Fixing the problem is squarely in your lap.
If you have an inexperienced caller, the caller won't know something's off, but the dance will be bad and end worse. Either the caller will stop calling and you'll keep playing or, even more awkwardly, you'll stop playing when the caller and dancers still have a figure to go.
I sometimes sing "Take Me Out to the Ball Game" with the lyrics shifted relative to the melody, starting the first note at "Me" instead of "Take," so the lyrics end before the melody. Same thing.
... take
Me out to the ball game, take
Me out to the crowd, buy me
Some peanuts and some cracker Jacks, I
Don't care if I never come back! So it's root,
Root, root for the home team, if they ...
There are probably half a dozen common reasons you'll wind up in this pickle. You might have picked a tune that wasn't 32 bars, then realized your mistake and switched; the caller might have dropped a figure once through the dance; the start of the dance might have been a little rocky and, by the time the dancers got into the flow of things, they ended up off; the fiddler might have played a part three times instead of two.
It doesn't matter why, you just need to fix it.
First, you need to be able to see there's a problem, then you need to be able to figure out how to start right when the dance does. If you're starting A2 when they're starting A1, the band needs to add a part. If you're starting B2 when they start A1, you need to drop one.
You have to know where the dance starts -- where its A1 is.
Watch while it's taught to see what the start looks like. Most start in only a few ways. If it's an improper dance, the first call will be something like "neighbors swing" or "neighbors do-si-do" or "neighbors gypsy" or "neighbors allemand left." If it's a Becket, it'll probably be "circle left, three-quarters."
When you're playing, watch the dancers, not just the band. When something's off, figure out what to do, tell everyone else in the band, "Okay, just one A next time through" (or "Three B's" ... whatever), and do it. Then check to see if you got it right. If it's not, re-calculate and try again.
If you forgot to watch, and can't tell where the dance starts, just stand up, walk over to the caller, and ask.
If you try but, for whatever reason, you just can't fix it, just let the caller and your bandmates know what the problem is, and be prepared to stop abruptly when the dance ends. If you all stop at the same time, it'll sound like you planned it.