After getting tired of a premade asset I used to synchronize the rigidbodies, I started writing my own module for doing so. How hard could it be?
First, what's even the problem?
We have rigidbodies in the game that respond to physics. For example, when a group of furballs stack, someone has to take care of them behaving realisticly.
If each client simulates physics, the world of each client would be obviously very different, so we need an authority.
So suppose the server is the only one controlling rigidbodies. It would then send messages to clients, updating where everything is.
But then responsiveness gets into the picture: say that the player hits the the jump key; they expect the avatar to jump immediately. It will not do to send the keyboard hit to the server, have it process the action, move the avatar, then update the client: the client in the avatar has to respond immediately!
So the server could control everything BUT the avatars. Each client would be responsible for its own avatar, and would update the server about its state. The server would be responsible for everything else, and would update each client about everything including the positions of all avatars.
Now a thought experiment: my avatar runs into a furball and pushes it. However, only the server has the authority to determine that the contact pushed the furball, as it is the only one computing physics. The server acknowledges the contact, but it has a slightly laggy version of the avatar, which is controlled by the client. Thus the server thinks the avatar is a bit back in the past, and its computed position of the furball is accordingly also not up to date with the avatar's position in the client. This effect doubles once the update about the furball reaches the client: since the server sent the update more time has passed.
The final effect is that from the point of view of the server everything is fine: the avatar is running and pushing the furball along. But on the client, the furball seems to be dragging behind the avatar.
Conclusion: everything the avatar touches needs to be responsive. For that to happen, each client needs to simulate the physics of objects near its avatar.
But wait! What if the avatar is pushing a furball, which pushes another furball, which pushes another furball and so on? For a coherent view, they ALL need to be controlled by the client, even though the last furball might be quite far away from it.
Furthermore, transferring control over objects both incurs a network overhead and is a chance for something to go not quite smoothly: there will necessarily be a moment when either both the client and server think they control the object, or none of them do. In any case, it is a moment where both of them try to smooth-out the transition from the control of one physics system to the other, which is bound to not agree on the precise state. Thus transferring control on-contact is not a good idea: contacts come and go by a frame-by-frame basis. You have to have a more stable mechanism of determining which is in control.
Lastly, it's probable that for a few moments two clients would think they have control over the same object. This might happen because objects that are not controlled locally are always represented on the client as they were in the past. So one client might think its avatar is closest to a furball, while another client thinks the same about itself. Thus, they would both simulate that furball's physics locally. It would be the server's responsibility to determine a "true" position which would be broadcasted to all other clients.
The premise of my solution is:
when an avatar is not in proximity to a rigidbody, it gets constant updates from the server on its state, and the client does not directly compute its physics. The client smoothes its own version of the object towards the state sent by the server. In addition, the client also extrapolates the last state received from the server according to the object's velocity and angular velocity, until a new state is received.
When the avatar does come near another rigidbody, it takes control and begins simulating its physics, and updates the server on its state. The server does the same thing the client did previously - interpolating and extrapolating - and updates all other clients.
When the object leaves the vicinity of the avatar, the client still claims control until a timeout has elapsed. If, within that timeout, the object comes near again, it ends up being controlled by the client for the whole duration.
When an object controlled locally touches another, the second object also becomes controlled locally (it is "infected" with local control), regardless of distance from the avatar. However, in order to prevent a situation where two close objects continue infecting one another with local control long after the avatar left the area, the infected object also inherits the timestamp of when the infector was last in the vicinity of the avatar. Thus after the same timout, the control of both is turned over to the server.
There are a few more pitfalls I tried mitigating. While not a problem-free system (for example - it is not cheat-proof in the sense that client decisions are not reviewed and verified by the server), there is also no industry-standard that works flawlessly for everything and everyone. I know that the ready-made module I used up until now was not satisfactory and had no way of supporting additional logic which is also applied to these rigidbodies. Consider: since the furballs have a mind of their own, but are additionally affected by physics, where should their logic execute? Should it switch hands between clients? Probably not: I'd like for it to always run on the server, even if a client controls the physics. My solution is designed to support this hybrid control. In this scenario, the server translates decisions made by the agent to physical actions (torques, forces etc.) which are sent to the current client that calculates physics. The client, in turn, incorporates this movement into the physics engine.
So this is the current state of affairs. In the video below, a red color means that the object is controlled by the client (the left window). I'm very satisfied with how the system achieves two contradicting requirements: on the one hand, physics are completely responsive in both points of view, and on the other - objects are very much in sync. The few hiccups occur when control is handed over from client to server or vice versa (this happens when an object changes color), but these changes are kept to a minimum by the infection+timeout mechanism.
Note: in the video the players pick furballs up and throw them. This is not animated as of yet.