The unlikely duo of Kevin Federline and Paris Hilton caused heads to turn in Las Vegas over the pre-New Year’s weekend.

On Sunday, the duo partied together for the second night running.

Both were in town to host separate New Year’s Eve parties – hers at LAX, his at Tangerine – but each name attraction started celebrating early, hitting LAX on Saturday and Pure Nightclub on Sunday.

A source close to Federline told PEOPLE that the two hung out in Hilton’s room Saturday after leaving the club.

Still, they seemed less chummy on Sunday at Pure, mingling only briefly before drinking a shot of tequila together.