Son's father in prison and now he's out and wants him.?

My son's father was released Feb 4 after being in prison for 6 years for felony drug trafficking. This weekend was the first weekend that he has gotten him ever. My son is 5 years old and so was born AFTER he was incarcerated. He is refusing to tell me where he is and refusing to let me have my child back. I have no papers because he has been locked up and I honestly didnt think he would do this. We were never married or anything. I called the cops out there to his cousins where he was this morning and they couldnt legally make him give him to me. I am going in the morning to try and get emergency tem custody. What am I going to be up against? I am scared that I wont be able to get it. I have never been through this before. I mean, he has only been out a week. If I do get custody then what are the chances that he will be able to petition the courts for custody or visitation and actually get anything worthwhile? Please help, I cant lose my son! His nae is on the birth cert