It's a good question because the booming internet age has really changed things in publishing, including the ability for agents to effectively pitch to NYC editors without having to be located in NYC.
That said, it does certainly help agents who are there -- if they're willing and driven to do the networking. There are many new "agents" listed on some of the rosters of prestigious NYC literary agents who have yet to make a sale after six months or more of being listed as agents. So they're either focusing solely on selling and managing subsidiary rights for the agents' existing clients (like foreign rights) or they're not being very aggressive about building up their own client lists and networking with/pitching to editors
We think that an agent's bio, career history, personal drive, and of course his/her sales track record matters more than anything these days. There are so many veteran NYC agents who are barely making any new deals any more because they don't need to... their client lists -- and their backlists -- are bank-rolling them in perpetuity. And so, they don't need to take on new clients, and even if you became one of their clients, it might not be a good fit anyway because if they're not selling on a regular basis, even those agents lose touch with which editors are buying what at each imprint.
Conversely, there are so many hungry agents with great established agencies (both located in NYC as well as outside of NYC) who are making deals on a regular basis that may be a better fit.
Like most things in this industry, it comes down to the competence and drive and connections of the people you are working with... not just a "blanket" yardstick of NYC agent vs. not, so it really takes a bit of analysis to really discern who the best agents are --> as well as who the best agents may be for you and your book.