I've done a fair amount of research on the line, most notably various threads at the Chesspublishing.com forum. "AMM", in particular, posted some fine analysis here:
Stefan Bucker had a good analysis on the gambit at Chesscafe.com but his articles went behind a paywall (and the site isn't looking healthy at present anyway).
I don't trust the gambit, but I can see why it is popular. Like the similarly dubious Englund Gambit (1.d4 e5), it has the merit of leading to unusual positions in many of the variations. The Svedenborg Variation (3.Bc4 fxe4 4.Nxe5 d5) often leads to considerable chaos with reasonable chances for Black. The main line involves a surprisingly strong exchange sacrifice: 5.Qh5+ g6 6.Nxg6 hxg6 7.Qxh8 Kf7. For this reason, I think White should avoid 3.Bc4, but it will be a natural reaction of many players who have not studied the line.
The variation with 3.exf5 e4 is probably theoretically better for White, especially in the case of 4.Ng1!?, where White argues that in this reversed King's Gambit, but Black can be satisfied with the attacking chances. 3.d4 is slightly better for White with accurate play, but Black has to watch out for a couple of dangerous piece sacs following 3...fxe4 4.Nxe5.
The problems are the simple 3.Nc3, which I think generally leaves Black a pawn down for just half a pawn's worth of compensation, and of course the main line, 3.Nxe5. 3...Nc6 4.Qh5+ and 3...Nf6 4.Bc4 are not looking too good, although 3...Nc6 might be worth a try in blitz games, since it generally leads to positions with insufficient compensation for an exchange.
3...Qf6 is probably best but it has a few problems. I don't like Black's position after 4.d4 d6 5.Nc4 fxe4 6.Be2 or 6.Nc3, although it is just about playable. After 4.Nc4 fxe4 5.Nc3, the 5...Qf7 line leads to dangerous attacking chances for White after 6.Ne3 and 7.d3, so I suggest that aficionados of the black side should rather try 5...Qg6. In general I think this is the hardest line for Black to face psychologically since it tends to be White who gets most of the attacking chances.
Is the Latvian Gambit refuted? It depends on how strong your definition of "refuted" is. I don't think it loses by force, but White certainly has a choice of ways to get a significantly greater-than-normal advantage out of the opening, and there are a few variations where I really wouldn't be happy with Black's position.
The illustrative games and analysis are here.
