So, I just reviewed my first academic paper. I'm not sure how much I'm allowed to say about it (since it's both pre-release and not my own work) and I can't link to it, but I will say that it was fascinating. It also was one of those unique experiences where you know you'll be repeating it countless times over the remainder of your life — but as this time is identifiably the first, there is considerable wracking of nerves.
I wasn't given a ton of guidance from the professor who gave the paper to me, which wasn't a problem but did leave some basic formatting questions, e.g.: Should my review be a typeset pdf or a plain text email? Was there any standard format to be used? I got most of this ironed out and submitted my second draft earlier today. I'm not sure how much of my review will be used / submitted back to the paper's authors. I've been lead to think all of it, but I haven't heard for certain.
Reading a non-peer reviewed (or at least not previously reviewed) paper was itself an odd experience. I'm so used to reading published papers (indeed, I think this is the first non-published paper I've read, save for those internal to my office) that the format itself lends quite a bit of credence to the subject, and at first I found the idea of criticizing the paper somewhat daunting. Here was a manuscript, identical at first glance to those I had read before, with all the requisite figures and formulas, and I was supposed to go poking holes in the arguments and thoughts of its authors. I had plenty of feedback eventually, but it certainly took a while to get going.
The image to the right is the logo of Springer Science+Business Media (the publisher of a number of academic journals) and was retrieved from http://www.nanonets.org/.