What to do in this messed up situation?

I think it's a scam and they've probably done this before. I'll bet you anything he was acting the whole time. The next time they show up tell them that they are not getting any money from you until his check clears your bank and ask them to leave your property. Legally you may be entitled to keep the money - you can check the landlord and tenant laws for your area. If you're legally entitled to keep it then simply hand them a copy of the law with the pertinent info highlighted and tell them to get off your property. If you tell them to leave and they won't or come back call the police right away.