We native English speakers have our prejudices. We see all those Dutch, German, French, Spanish, Portuguese, Scandinavian or Israeli people speak fluent conversational English and feel like English should be an accessible language to everyone.
Let first set things straight. English is from the Germanic family of languages, which includes Yiddish, Dutch, Norwegian, Swedish, Danish, German and many other languages. Those languages have similar grammatical structures and their basic vocabularies have the same roots. As for French, Italian, Portuguese, Spanish and Romanian, they are Romance languages, and English has borrowed many many Latin and French words throughout the years. Note that modern Hebrew has lots of borrowings from Yiddish.
Speakers of these languages are the foreigners Americans are the most likely to meet, as whether they are Latinos, Africans or Europeans, there is a big chance such people speak one of those languages related to English or from which English borrowed significant portions of vocabulary.
And then there are the Asian, who borrowed some vocabulary from English, but not enough to make learning learning English easy. While Dutch and German people can easily infer what an English word means by looking at their native languages, Koreans can't.
Koreans have this disadvantage compared to speakers of other languages. And with the prejudice English speakers have that most foreigners have a language related to English in some way, we think that Koreans are slow at learning English.
While Dutch and German people may still make "cute" grammar mistakes when they speak English, they are not afraid of the "unknown": if they hear a word they never heard before there are high chances that such words will be in their native language. That is not the case for Koreans.
Koreans therefore go through what linguists call "language shock". Just like culture shock, when people experience language shock, they refrain from speaking the language that they are learning from fear of not understanding what the person is saying and being considered "stupid". Native English speakers learning Korean also have their "language shock" phase.
The rest is a question of motivation and need: Koreans who need to learn English to communicate with their husbands will speak English better than those who need to learn it to get a decent score at the TOEFL test which will guarantee them a job.