Moving to Lausanne - English speaker with baby!

Dear forum members, my husband has just secured a new job in Lausanne. He speaks fluent French but I'm afraid I only have schoolgirl standard (I'm English). Of course I will try to improve my skills, but how difficult will I find it in the meantime?

Secondly, we've lived in various different countries for the last 4 years and usually I have no problem finding things to do and joining social groups etc. However, since our last period of living abroad I have had a baby, so many of the things I used to do I won't be able to manage as she will always be with me. I'd be really grateful for any suggestions for social groups/activities in the area that would be accessible to me.

Hope you can help!
Thanks,
Helen