Legal Question in Family Law in Maryland

Since you don't have a child custody court order (I assume), he could legally do this. This usually happens because the father is afraid of getting hit for child support. You need to consult an experienced family law attorney to prepare a custody and support agreement which will also provide for visitation privileges for the father. If he's not willing to agree to that, you better get a court order. If expenses are an issue, check with your county bar association about pro bono (free) and reduced fee attorneys for whom you may qualify.