Until you start speaking to someone from that part of the world.
Much like a Korean or Lao might get a bit "Huffy" if asked by someone, "Are you Japanese or Chinese?" 
Or a Welshman who is mistaken for a Brit.
Or a Kiwi who is mistaken for an Aussie.
There's a good reason why Canucks have Maple Leaves sewn into everything they own when traveling abroad. Especially some parts of Europe.
*laughs* all above points are valid

A) My gene pool from Grandparents down includes 5 nationalities, 3 religions and a partridge in a pear tree. Ergo, I have zero moral rights to expect anybody to know where I'm from by looking at me.
B) Accents change, and I don't expect anyone from the other side of the world to really know the difference between an Oz vs Kiwi accent (or British (inc. variants), or State/locational Aussie derivatives). Such expectation leads to madness.
Breaking labels down to a continent has always been the safest options when travelling: ie
- European (vs stepping on the toes of Italian, Greek, Spanish, Swedish, French etc)
- Asian (vs North Korean, South Korean, Chinese, Japanese etc)
so and so forth.
...leading me back to that diverse label of
American. You guys are on a big continent and all sub-continent labels are just directional derivatives..ie:
- The continent of America is broken into North, South and Central.
Therefore IMHO, while I respect that it's a big area with diverse differences between residents, anybody from that continent who gets iffy at the general term 'American' needs to re-check their map
