If you've spent most of your career being an employee, you'll find that communication is different than a boss to an employee. The funny thing is that while many become freelancers to "be their own boss", the truth is the client is the one giving orders. There's usually a happy medium that makes client-freelancer relationships work but sometimes, this isn't the case. Some freelancers can become overzealous when it comes to their beliefs (which sometimes may be based on a hunch) and some clients can just be...not nice to work for.
Don't Hide Your True Feelings at the First Sign of Trouble
While most in either position may not have a problem delegating their wishes, this is where clarity comes in handy. Do you feel the job tasks do NOT justify the pay? Is the freelancer working too slow? Is there a way the two of you can work without giving more money to an outsourcing company that offers little to no support (not naming names here...you fill in the blank, LOL)?
Observe Patterns and Rectify Immediately
In the remote working world, you can arrive stinky or in need of a pedicure but if your sick cat is getting you down, then time off may be necessary. Taking a short leave early in the game (but try to finish remaining work first, especially if it's a new client) with a brief explanation is better than late and/or incorrect submissions, lack of communication, or being verbally aggressive with the client. If the client goes elsewhere, you can at least say that the parting wasn't based on faulty work. In most cases, people understand that life happens.
Encourage Open Communication From Day One
For a lot of people, being over something is just that...life moves on because you can't succeed while wallowing in the past. Even if memories are fresh, a person that respects their business operation won't suffer because their current freelancer is hard to reach, rude, unpredictable - or all of the above. While some of you may fear a talkative client with the potential to pay many future bills, there are ways to handle conversation without seeming ungrateful.
Getting a feel for what a client may be thinking about you can take time. Although some people find surveys to be impersonal, you may want to send out a short questionnaire to all clients on a regular basis to find out their true satisfaction levels. If there's an opportunity to remain anonymous (or at least offer some reward for completing the form), they may offer the constructive criticism you've been needing.