It would not be considered impolite, but it's not exactly a greeting either. I mean, you don't go around waving at English speaking people saying just "Friend!", do you? People will get the impression that you're intellectually disabled if you do that. You can say "Hola, amigo!" to a male neighbor (or "Hola, amiga!" to a female neighbor), which just means "Hello, friend!". Remember, the "H" in "Hola" is silent.