I think it depends on how and where you learned the language, what your priorities are, how confident and sociable you are and what your resources are. When I was in Korea, I found it easy to learn how to speak Korean because I met so many people who spoke worse English than I do Korean. It forced me to try to make myself understood in Korean and I made a lot of progress trying to make friends with people who didn't speak a lick of English, German, Italian or French. At first, our conversations were really, REALLY primitive and I can't say they are complex now or that I'm in any way fluent, but chit-chat about family, school, friends, hobbies, likes and dislikes etc. I can comfortably do.
Like I said, I think small-talk is comparatively easy to learn in any language. You don't need a large vocabulary or advanced grammar for it, all that's required is having the opportunity to speak it and, maybe most importantly, not being afraid of making mistakes and speaking with a certain confidence. Getting to the level of being able to hold complex or abstract conversations, though, that's a different matter and probably takes 10 times as much time as learning how to hold everyday conversations. I think that, unlike reading articles or understanding dramas, easy conversation is less a matter of skill but more a matter of confidence and opportunity.
I think conversation can be learned faster because new words will stick with you better - I think it is true for most people that they will be able to memorize words and grammar patterns better when they are said in a fun conversation than when they are studied from a dry textbook. However, it is true that getting over the initial shyness of speaking the language can be a hard and intimidating task. When I started speaking Korean, my boyfriend laughed at my pronunciation. Then, I threatened to hit him if he laughed again (

) and continued to talk to him in Korean. I got the gist of it soon after that, my pronunciation got much better very quickly and I was able to memorize certain speech patterns better because I could associate situations, pictures and feelings with them. And then I went to Korea and made friends with Koreans who spoke no English and now my spoken Korean by far beats my written Korean.