Little guinea pigs can become very cold very quickly. Have you kept the room they are in warm, when Meribel gave birth I constantly had the heating on in the room the were in?
Can you offer them a hot water bottle or snuggle-safe wrapped in a towel or fleece to keep them warm/drape a towel or blanket over one side of their cage, offer them a deep bed of hay to hide and stay warm in? Mum won't stay by them all the time to keep them cosy so it's very, very important you keep them from catching a chill.
How is mum with the babies, has she been feeding them (have you seen them suckling?)