Legal Question in Real Estate Law in California

First off, the city does not permit the location of any structures. All the permit process does in theory is make sure things are built to code. In reality, though, it doesn't even guarantee that, since cities are exempt from liability for code violations they do not catch. So ultimately the permit process is just a zoning enforcement system and a construction tax. Unfortunately as between you and the neighbor you complicated things by building a wall instead of replacing the fence with something pretty much the same. Had you done that, you could have sued him for 1/2 the cost. Had you built it entirely on your own property, the neighbor would have no rights of any kind. Having built an encroaching wall, you have a situation that is WAY too complicated to resolve in an internet Q&A forum.

A block wall may be enough thicker than a wooden fence to intrude an additional 2", even when its center line is exactly the same as the center line of the old fence. If this went to court, it's anyone's guess what the judge would rule; they do have some latitude to find that you made an innocent mistake and only award the neighbor some damages, but there's also a good possibility that you'd be ordered to remove the wall. You might want to consider what it would cost to have a survey made to determine whether the neighbor is correct about the 2-inch encroachment. Maybe it's less, maybe it only encroaches at one end, maybe it doesn't encroach at all. The city and the permit have nothing to do with whether the wall is encroaching.