You'll get benefit from the new page once it has been indexed.
The other site should have redirected to the page that was already indexed.
The URL that now redirects will soon drop out of searchengine listings if it is a 301 redirect.
Using a meta_refresh is a bad idea. If the other site cannot do a proper 301 redirect, using rel="canonical" would have been a better option.
You should check the new page using "View Source" and make sure there is not a rel="canonical" element pointing to the URL that now redirects. If there is, then you have an infinite loop. In that case, Google will either ignore the technical error and list the page, OR will become confused and remove both copies from their index.