Aubry was dropping four-year-old daughter Nahla to her mother's house when he reportedly became aggressive towards Martinez.
Sources say the male model "went completely nuts" at the French actor.
He threw punches as shocked Berry (46) looked on.
But Aubry (37) wound up pinned to the bonnet of a car by the time police arrived and booked him on battery charges. It has not yet been confirmed whether Nahla witnessed the incident.
Olivier (46) had reportedly approached Aubry with a peace offering. As he walked towards the model, he said: "We have to move on", which is believed to be a reference to the recent custody battle which blocked Halle from moving to France with Nahla.
But Aubry responded by pushing Martinez away and attempting to clock him in the face.
According to onlookers the fight was 'brutal'.
"Olivier blocked Gabriel's punch and the punch struck him in the shoulder instead," they said.
"Gabriel then pushed Olivier to the ground, a struggle ensued, ending when Olivier pinned Gabriel to the ground."
The cops appeared shortly after the scuffle and took Aubry away. He was later released on bail for more than €15,000 and is scheduled to appear in court on December 13 in relation to the incident.
A judge also issued an emergency protective order -- meaning Aubry has to stay 100 yards away from Halle, Olivier and Nahla.
Halle and Aubry split in 2010 after dating for four years. The pair have been involved in complicated custody dispute over their daughter since the relationship ended.
This reached a head when Berry expressed her intention to move to France with her daughter and Olivier. Gabriel had petitioned the court to stop Halle moving.
Aubry said it would give him limited access to his daughter.
But the Cloud Atlas actress countered that the privacy laws are stronger in France and would keep Nahla safe from the paparazzi.
In reference to her custody battle, the star said earlier this year: "There is turbulence, but this too shall pass".
Apparently, Martinez has become increasingly fatigued with Aubry's actions over the past year.
Olivier and Berry even put their wedding plans on hold earlier this year while the actress focused on her custody battle.
Both Martinez and Aubry were taken to hospital following on from the fight. Martinez was seen entering with a bruised and battered face and a possible injured hand. It has been reported that Aubry is suffering from a broken rib, bruises to the face and head injuries.