Well, I think I found the real problem.
I have created all the appointments with only minimal info. That means I have added only required fields (subject, start & end time, owner). The organizer has not added. And that is the problem.
When you don't add organizer, the appointment syncs ok in the first time, but after that the error occures until you add the organizer. So this is not related to my custom filter or team in the owner field.