Add a file splitter to handle padding

I have written a splitter that relies on the media server and also can for legacy purposes use a file as a source. My reasoning for why I would task the media server with this activity is because SageTV is cross-platform and this is the only place where everything can easily meet up. I also like the tidiness of of the recording stream going into memory, then being split off into multiple files as opposed to have a primary recording and chopping random bits off to append them to other recordings or using multiple files for each recording which while definitely the most efficient use of space will be very frustrating for both Comskip consistency and people using external transcoding for archival purposes.

The way this works is very simple on the surface. Essentially SageTV gives the encoder a unique filename (integer in hex + .extension) in the path of a real recording directory. This is referred to as a virtual file. It places the file with it's upload ID into a map that's checked very similarly to how the current upload ID process, but is can tell which one it is looking for with absolute certainty even though the connection commands are exactly the same. If it is not known if the encoder will use the media server or not, a monitoring thread is also started for the file that might be created. If the encoder connects to the media server, the monitoring thread if running is stopped immediately. If the file is created and becomes larger than 0 bytes, the monitoring thread will read the file as if it was data coming in through the media server and split it out the same.

When the monitoring thread is being used for input, it will first read and split everything that's in the file at that moment, then it starts to try to calculate the rate that the file is being written so it's not constantly banging against the filesystem. It tries to stay about 64K off from the very end of the file, but it can read and split all the way to the end of the file if the rate fluctuates a lot. The calculation is also regularly adjusted so we don't end up falling hopelessly behind. Over a gigabit connection using SMB I was able to split a 20Mbps stream into 24 files. This is of course well beyond any reasonable usage, but it's good to know it can be done. When I tested locally, I was only limited by how fast my storage is. That same 20Mbps stream was able to be split into 200 files before I started to have problems keeping up.

The monitoring thread is only here as a shim for current network encoders that do not use the media server for various reasons. Everything built into SageTV and network encoders that use the media server will be using this splitter efficiently when this to be beta feature is enabled. The only network encoders that will be excluded from this feature will be ones that don't support SWITCH. I haven't decided if we should try to handle transitioning in place of the network encoder having the ability or not. It could be fairly easy when it's a known format and we already have decent code for it in the media server remuxer.

When the media server is used for input, the file opening is validated just like a normal upload request and it creates a normal FileChannel that only appends to the files it is has been told to split into. I initially added the ability to set positions relative to the first virtual file, but I scrapped it because it was adding additional calculations to every single write and encoders in all cases that I can think of only append data. This is compatible with the media server remuxer too and requires no modifications for it to split the file correctly or transition.

Any time we need to add files or stop writing to files, the encoder is signaled to SWITCH and it is given a new virtual file. As soon as it closes the old virtual file, all of the files we are removing are closed and all of the files that are still being recorded remain open. When the new virtual file is opened all of the files that remained open resume and all of the new files start writing. After the encoder returns that the SWITCH is done, the old virtual file is immediately closed so that no further writing to the splitter can happen even if a media server connection was left open (it will throw an exception). This allows the network encoder to decide when is a good time to transition because it might have a different or better idea of when/where to do this over a what SageTV might determine for it. In testing adding and removing between just 10 files, 200ms was trimmed off just by leaving files open when appropriate through transitions as opposed to closing everything and re-opening everything as appending for the new virtual file. When stopping, you would just tell the encoder to stop and when it returns close the virtual file.

These virtual files are designed so that you can also stream anything to the media server with the same base name (integer in hex) with any extension and it will allow you to write that file and have it split the same as the only file that is actually expected, just with a different file extension. Of course you need to write the A/V stream to the virtual file + extension that it was expecting, but you can also use that virtual file name to stream subtitles that can't be muxed into the other stream for example. This feature was probably one of the bigger pains to design, but I felt like it was a bad idea to do a fixed one to many mapping because we would likely want this flexibility at some point in time anyway.

I now need to work on Seeker to get it to understand how this all works and probably a few small changes to Scheduler. I'll be using the remove back to back padding option as suggested. I'll figure the rest out as I go.