Non-religious/ athiests in nigeria (Lagos)

Hi everyone :)
In a couple of years I'll be relocating to Lagos with my Nigerian husband. Im not religious and I'm worried it will be a problem for me! I know Nigerians aren't to fond of people who don't worship a god. I have no problem with religious people (my husband is Christian) but I don't want to get a bad name for myself by not attending church on Sundays! Any advice/tips?