Since I now have the job of cutting (cropping/clipping) the A/V recordings of the more than 200 presentations of 21C3, I've been looking for a number of days for available free software to do GOP / key frame accurate cutting of MPEG2, mp4 and OGG/Vorbis files.
As for OGG/Vorbis, the vorbis-tools package contains a program called vcut, which basically does almost the full job. However, it's a bit clumsy to use, since it always splits a original file into two halves, before and after the cut position. I've modified it a bit in order to accommodate my needs better.
As for combined audio+video containers such as MP4, it becomes a bit more difficult, since you need to find key frames for both audio and video as close as possible to the user-specified cut point.
However, after learning a bit about Apple Quicktime and the MP4 container, plus the help of libmp4v2 from the MPEG4IP package, I was able to create a small tool for key-frame accurate cutting, too.
For MPEG2, there is lve (Linux Video Editor). This program even provides a graphical user interface for navigation through the video, creating clips and a cut&paste interface. Unfortunately the UI is not intrusive in any way, and it even seems to use it's own toolkit. After playing with it for more than 45 minutes, I wasn't able to actually cut a single video using it :(
Since MPEG2 is not a priority at the moment (we need to make .ogg and .mp4 available for download ASAP), I deferred this problem for now.
Maybe at some point I'll find the time to put together all the pieces and create some generic media cutting/clipping/cropping tool for any kind of format. However, judging from the differences of the media formats, there wouldn't be much more common code than parsing the command-line options ;)