Do we have a case against this person?

All of your stories about the neighbor are irrelevant to this case. This case is about your dog being a nuisance. You need to get control of your dog. If you have anything else you want to make a legal issue out of, then that belongs in a separate case, but holds no weight with regards to you not securing your dog.