Neighbors

Matthew 22:39 says to love your neighbor as yourself and I'm starting to wonder why I pick my neighbors! When I need something who do I see? I tell myself it is my neighbor I'm going to see when in reality I go to my friends. More than that I go to my neighbor who I'm willing to expose myself to. So who is my neighbor supposed to be? Do we get to pick and choose who our neighbor? Or is it supposed to be our literal neighbor?