--------------------------- --- The Yatta Manifesto --- --------------------------- ------------------ Prologue & Premise ------------------ Not much of a manifesto but hey, the name sounded cool at least. Couple things before diving in. If you have no idea what you're doing at this point you probably shouldn't be reading this or be anywhere near this page. Knowledge of video stuffs (IVTC, etc...) is a prerequisite to yatta. If you find yourself asking "Why?" to a bunch of things in this document then you might be getting a little ahead of yourself. The whole point of yatta is to improve upon the decisions made by tfm/telecide/tdecimate so if you don't really know what you're doing in yatta you're probably better off just using tfm().tdecimate() or else you might unknowingly fuck shit up and make it worse that it would've been. The purpose of this is to provide a practical usage guide to ymc and yatta. I may refer to pattern guidance as PG in this document. --------------------------- General Yatta Usage and You --------------------------- The ideal yatta session will involve very little to no use of the mouse. You should try and do everything with the keyboard and keyboard shortcuts as they make things go much faster (and easier). So go look at the hotkey list! Some random general facts about yatta; A lot of the stuff in the right click menu is probably useless. A lot of stuff in yatta are 'one time use' features which were made for a niche case and will probably never be used again. In my opinion "Try Pattern" is useless. I think the set pattern (ctrl+t) is quicker but this is just personal preference (although I think the majority would agree with me). Good hotkeys to know: Ctrl+T - Set Pattern Dialog Ctrl+S - Change Pattern per section Ctrl+Q - Delete current section Shift+Q - Delete current custom list applied P - Postprocess current frame Ctrl+P - Postprocess section (or remove postprocess from entire section, useful trick!) Ctrl+Alt+G - current section pattern guidance (local pattern guidance) I - Start section at current frame O - start/end of no decimate sections C - start/end of custom list F - freezeframe Q - Delete freezeframe S - Change match of current frame Ctrl+Up - Next Section Ctrl+Down - Previous Section Shift+Right - Forward 10 frames Shift+Left - Back 10 frames Up - Next vthresh hit Down - Previous vthresh hit M - Next postprocessed frame Go look at the settings dialog where all the hotkeys are. Learn them all, become a god. ------------------ Metrics Collection ------------------ DVD2AVI or DGIndex? Use whatever the fuck works. DVD2AVI will provide faster seeking if it works. timecop's modified to accept 188-byte transport streams: http://pbx.mine.nu/dvd2avi/ For metrics collection you need a shiny d2v file to throw at ymc. So pick your poison (dvd2avi/dgindex) and make one. Order of filters: (Cutter->Crop->)tfm+telecide metrics->SCXvid->tdecimate Bare minimum needed for pattern guidance: Telecide->SCXvid->Decimate Crop and Cutter are both optional and should both be done before matching. Cutter is generally used on transport streams to cut commercials and 99.99% of the time don't use it on dvd's. Cropping before matching can help if there's shit on the edges that could mess up the metrics (cropping helps get better metrics). Feel free to be liberal with the crop and overcrop, this is just metrics collection and doesn't matter that much. The whole point is to remove shit around the edges that could mess with metrics collection so undercropping here (or anywhere really) is retarded. TFM+Telecide Metrics will produce better matches for when pattern guidance fails or for some reason you want to resort to the original matches in yatta. Couple notes though with regards to it's usage. You MUST change the field parameter so that you get c/n matches (instead of the default c/p). While theoretically the same yatta needs c/n in order to properly perform pattern guidance (which uses cccnn permutations). Order 0 -> Field T Order 1 -> Field B You don't have to touch anything else in tfm's settings. You can set slow=2 if you feel like donating more cpu cycles to tfm. I'd recommend keeping micm/mode to 0 so you don't wind up with a p match in the middle of a high motion on a shit video because tfm is retarded enough to put it there. If you're planning on doing pattern guidance NEVER run ymc with just 'tfm'. Yatta needs the metrics from telecide in order to do pattern guidance. Using just Telecide instead of tfm+telecide metrics isn't recommended but should you chose to do so (hey it's faster) you don't have to touch anything inside the telecide settings dialog. SCXvid has to occur after matching but before decimation. The reason for this is because you need scene change information BEFORE decimation since that is how the pattern changes. TDecimate/Decimate should be self explanatory in its positioning in the chain here. Just keep in mind that this has no bearing on the final output of yatta, this is just for metrics collection. Hit start and twiddle thumbs for 5 to 30 or so minutes. --------------------------------------- Presets, Custom Lists, Sections and You --------------------------------------- Sections: When you open up a yap file after running ymc, sections will already be in place that were automagically generated from SCXvid. Every frame SCXvid flagged as an I frame (keyframe) will get marked as a new section. Sections are scene changes as detected by SCXvid. In the sections list in the tools menu only the start frame appears but keep in mind that end frame is the start frame of the next section subtracted by 1 (start and end frames appear when you're working with the video in the top left). Presets: Can be applied to the whole video by calling them bare at some arbitrary location in the script or can be applied on a per section basis. They're defined in the tools->preset menu. In the most basic sense it's a collection of one or more avisynth filters which will get applied wherever you put them. Remember to hit "update" every time you change a preset. Custom Lists: Similar to presets except they are not bound by sections and are applied to an arbitrary range of frames. Unlike presets, they can only be on line (preferably a single function call), and they must apply the frame range themselves using %s and %e, which are the start and end frames respectively. For example: myfunction(%s,%e) which applies itself to those frames. In the tools->custom lists menu, the left box is a list of your defined functions, the top right large box is the list of frames to which it is applied, the middle right box is the function call for the selected function, and the bottom right box is the filename to which you have saved it to (which is completely useless so don't even worry about it). The most common example of this is reblend or reblend_vin as far as I know and is the only practical usage of custom lists. Last but not least - LEARN THE FUCKING HOTKEYS!!1111ONE ----------- Yatta Setup ----------- Couple things before you get started in yatta. Some of these can be set once and you'll never have to do it again, others you'll be doing every time you load a yap file for the first time. To start off in the settings dialog there's a setting called "show too short warnings". This is where yatta usage can branch a little big because of discrepancies of whether or not ignoring too short sections really makes a difference. The thing about too short errors is that when they occur it is an inherently high motion place in the video, which means it should be safe to ignore too short warnings. So if you're feeling a bit lazy you can uncheck it and ignore them and no one will be the wiser. However this is yatta, we manual IVTC everything to the extreme so the only way to guarantee no fuck ups is to not ignore them. They don't increase work load a lot it's just a bit of extra work (it'll appear as though there's a lot of them but they're often grouped together and can get plowed through rather quickly). *DISCLAIMER* It's probably safe to ignore too short warnings due to reasons above (high motion) but ymmv. *END DISCLAIMER* Tools Dialog: AVS Generation: Tdecimate spline36 and nnedi3 are decent but pick what's best for the situation Find: VMetric from both sides 5-10 (I guess?) for dvd, 30 for transport stream (these values are in general, may depend on the source but they're good starting points) Here's where old sort of meets new. With hard telecine comes combed/interlaced fades (aka telecine combing, scanlated fades). This occurs on fades to and frame black or white and on cross fades. The old way of dealing with it was to use a custom list and the reblend fucntion. This basically blurred the fade till no tomorrow to get rid of the combing. A vtresh search (explained later) will pick up 99.99% of these but going through the video doesn't hurt to apply these via a custom list. A newer way of dealing with it is a modified version of reblend which makes use of the excellent vinverse plugin to deal with the combing. Available here: http://warpsharp.info/yatta/reblend_vin.avs If you are using reblend_vin then add a custom list entry and call it whatever you want (right click in lists, new list -> empty). Click on the new list and in the first of the two text boxes put "reblend_vin(%s,%e)". Right click your custom list and put it as "post telecide". I don't exactly trust anything that does automagic thresholding but vinverse() by itself does have fairly good thresholding built in so another way to deal with it is to have a preset with just vinverse() in it and set it to apply it post telecide. We're done in the tools dialog for now, right click anywhere in the video Additional->Cropping and Resize. Set your final cropping and resizing here. HIT SAVE and don't save again till you're finished with pattern guidance. Once again right click anywhere in the video Pattern Guidance->Set Min. Length and set it to 5. The default of 20 is too long. 10 is safe, real men use 5. --------------------------------------- VFR (setting no decimate the right way(tm)) --------------------------------------- Setting no decimate on 30fps sections should be done prior to pattern guidance. The reason behind this is that you can then set no decimate on the entire section without those pesky decimates getting in the way (because yatta needs stuff to occur mod 5 you wouldn't be able to set no decimate until 5 frames after the last decimation which may not cover the entire vfr section, however this probably wouldn't be noticeable on playback but it's nice to be right). There are three ways of searching for 30fps sections in yatta: #1: VFR Search 1: Search one looks for dmetirc values set by a threshold. The threshold is defined in the "Find" tab in tools, in VFR -> Set threshold. I have no idea what a good values is for this since I don't use it. It searches for where the demetric between frames is lower than this value. I don't see much use for this type of search. #2: VFR Search 2: This is more useful than the previous. It searches for 'n' c matches in a row as defined by the same 'set threshold' button in the VFR part of the Find tab in tools (even though it's labeled as dmetric the same value will be used for both). It may be a good idea to pump up this value a bit. #3: Ctrl+Up and use your eyes: Ctrl+Up and look for a bunch of c matches. Once you've found your sections you would like to no decimate, in the set pattern dialog (ctrl+t) set matches to 'c' and decimate to 'k' and then ctrl+s over the sections in question to set them all to the proper c matches. Then at the start of the place you'd like to mark for no decimate hit o, go find then end (probably ctrl+up plus back one frame) and hit o again to mark the section for no decimate. ------------------------------------------------------------------------- Pattern Guidance Non-Shitty Transport Stream Way (your source is probably a dvd or a clean hard telecine transport stream) ------------------------------------------------------------------------- It's time for magic and for you to lose your sanity. Remember this is a practical usage guide for ymc, yatta and pattern guidance. It will cover the most common types of errors and sources you'll probably deal with but I'm not psychic and every source is different so ymmv with how useful any of the following is. Right click video -> pattern guidance -> pattern guidance -> are you sure? Fuck yea. And with that you'll be presented with a log, a log of all the nasty places everyone would rather ignore. So now that we have matching failures what exactly do we do with them? Well it's sort of obvious if you understand patterns but this should provide information on common reasons for a matching failure and how to deal with them. The first thing to understand is that matching failures go the end of the ORIGINAL failing section so be sure to keep that in mind as you might add sections. Common Reasons for a Matching Failure: * Bad scene change detection (oops SCXvid missed something) * Fade to black/white and then from black/white * Pattern change in the middle of a scene (fun game, guess where the pattern change occurs) * Metrics get kind of funky in pans (get your funk on) * Not enough info from metrics for PG (really low motion) * Plain old fucked up source (even yatta can't work miracles) * Crossfade (two overlapping patterns, annoying as fuck sometimes) The following is going to be a sort of step by step guide into fixing the failures defined above. I'll go through each one by one and common ways to fix them. Remember there is always more than one way to do things and I may only cover the most common way (or most sane way). Some of the ways I go about things are personal preference although if there is multiple ways of dealing with something I will try and provide some insight. *ATTENTION* Please always look through the entire original failing section before doing anything to understand WHY it's failing. If you don't understand why a certain section is producing a matching failure you're doing it wrong(tm) and this guide is pretty useless huh. BAD SCENE CHANGE DETECTION: Fixing this problem is pretty straight forward. This means that SCXvid missed a scene change and there's two scenes with two different patterns but they're not sectioned off correctly. Go through the failing section and look for it, once you found the scene change press 'I' to create a new section at the start of the new scene SCXvid missed. If that was the only problem and missed scene change you can go back one frame, ctrl+alt+g (pattern guidance for that section only), and forward one frame and do the same to (re)run pattern guidance on our now correctly sectioned off two sections. If for some reason one of the two halves fail then look for the reason why (could be any of the reasons listed at the start of this section). FADES TO BLACK/WHITE AND THEN FROM BLACK/WHITE: This sort of falls under the same category of bad scene change detection but there's no way that SCXvid would've been able to pick this up. It should be obvious why it's causing a matching failure. There are two different patterns, one before the fade to black/white and a different one when it fades from it. The fix is also really simple. Just put a new section in the middle of the fade (you can use the metrics to find the best place to put it if your eyes are broken). Then just as the bad scene change detection run your local pattern guidance on the two separate sections (ctrl+alt+g on both). PATTERN CHANGE IN THE MIDDLE OF A SCENE: Sometimes with hard telecine dvd's and what have you there may be a point within a scene where the pattern arbitrarily changes. Your goal as the yatta user is to find the pattern change. After a while of doing this it sometimes because easy to just spot it by eye (for example one character may stop moving and another may start moving, or something about how they're moving may indicate it but that's just kind fun guessing). Fool proof way to figure it out: ctrl+t -> cccnn kkkkd -> ctrl+s at the start of the section and advance until you spot some combing. At the frame you spotting more combing at ctrl+s until the combing goes away. Go back to the beginning of the section with ctrl+down arrow and make sure it didn't fuck up the start of the section. Basically this is to obtain the first pattern. Once you're fairly certain you figured it out just look for where the scene starts combing again and there's your pattern change. Create a new section at this point and try to run local pattern guidance over these two new sections. If you didn't get any new failures then you're good. If so you picked the wrong place OR there may be more pattern changes. Unless you're fucking pro and can spot patterns in the matches a mile away or can see it in the 12 or less fps motion that's going on sometimes just keep at it till you've the pattern change(s). It's a relatively easy thing to fix so if it's causing you trouble you're doing it wrong(tm). *LAZY MODE WARNING* If you're lazy you can probably just ignore these. Since the matches are all correct TDecimate will probably do a sure fine job of decimation 99% of the time but this is fucking yatta so do it the right way(tm). *END LAZY MODE* METRICS GETTING FUNKY DURING THOSE PANS: Sometimes metrics get a little out of wack during 24fps motion pans. Most of the time the motion will seem smooth and you'll question how exactly this caused a matching failure but sometimes it's because the vmetric gets up there with pans. Other times it's because it's a really slow pan which also causes some metric confusion. Considering that most of the time likely the only problem is that yatta barfed on the metrics so it's really easy to fix. Find yourself the first of the two dup frames. ctrl+t to bring up the set pattern dialog. cccnn -> -> kkkkd -> . ctrl+s until the decimated frame is the current frame. You can check around to see if you got it right. It also may not be a bad idea to check the last frame in the section (ctrl+up + left arrow once) to see if it's combed but if you don't do that you'll catch it later during high vmetric search anyway. REALLY LOW MOTION (SQUINT TO SEE THAT MOUTH MOVE WAY OFF IN THE DISTANCE): This is basically "find the right pattern". So ctrl+t -> cccnn -> -> kkkkd -> . ctrl+s at the beginning to set our first pattern, advance until you see any combing. ctrl+s until that combing is gone and so one and so forth until there is no more. Congrats you found the right pattern, easy right? CROSSFADES (THOSE ANNOYING MOTHERFUCKERS, TWO PATTERNS ON TOP OF EACH OTHER WTFHAX): Easily the most annoying thing to deal with on hard telecine DVD's. It's a fade where there are two scenes overlapping, so their respective patterns also overlap. You can see how this might be a bitch to deal with right? You should always section off the crossfade from the rest of the stuff. So you're left with three different sections (beginning with one, the failing one): 1.) Stuff before the crossfade 2.) The crossfade 3.) Stuff after the crossfade This would be sectioned off something like: New section at start of crossfade, new section at end of crossfade plus 1 frame. You can run local pattern guidance (ctrl+alt+g) on 1) and 3) leaving you just with the nasty bit which is the crossfade itself. There are a few different approaches to dealing with it. There are basically 3 different types of crossfades. Two are easy to deal with and the third is a bitch. 1.) Crossfades where one or both halves is static. 2.) Crossfades where the patterns match up (doesn't even show up as matching failure!) 3.) Crossfades where there are two different patterns overlapping Furthermore the 3rd can be broken down into another subset: 3a.) Crossfades where one or both halves has 12fps (or less) motion 3b.) Crossfades where they both have 24fps motion You could run into something which doesn't fall into any of these classifications (evul 24fps overlapping with 30fps?) but the above are the most common. These various types of crossfades can and should be addressed differently. All crossfades except 3a) and 3b) will be 'perfect'. 3a) might have a wrong decimation somewhere but will still appear fine on playback. 3b) is the only one where you may run into trouble with stuttering on playback which is what we try to minimize when doing it in yatta. #1 - CROSSFADES WHERE ONE OR BOTH HALVES IS STATIC: You can section off the crossfade so the section in motion isn't sectioned off from the crossfade. Then by running local pattern guidance on it it'll apply the right pattern for the crossfade itself (then do the stuff after/before the crossfade depending which half was in motion). You'll still have to reblend/vinverse the crossfade to deal with the telecide combing. If both halves are static and you're asking why does it show up as a matching failure you don't understand how patterns work. The stuff before the crossfade was one pattern and the stuff after the crossfade was a different pattern. Since it's all one section it can find a common pattern accross the entire original failing section. #2 - CROSSFADES WHERE PATTERNS LINE UP: This won't show up as a matching failure so thank whoever made the dvd. #3a - CROSSFADES WHERE ONE OR BOTH HALVES IS 12FPS (OR LESS) MOTION: If one halves of your crossfade is 12fps motion (or less) and the other half is full 24fps motion then set the pattern of the crossfade to that of the 24fps motion section. Then postprocess the crossfade with something like nnedi3. If both halves are 12fps motion or less then you can just section it off and apply local pattern guidance to the surrounding sections and just reblend/vinverse the crossfade itself and leave any patterns alone. #3b - CROSSFADES WHERE BOTH HAVE FULL 24FPS MOTION: One of the hardest things to deal with in a hard telecine DVD/transport stream. There are two incompatible patterns overlapping each other. At a minimum the crossfade should be postprocessed. Dealing with this kind of crossfade can come down to fiddling with matches and decimation to try and make the final result as smooth as possible. Sometimes what you get is the best there is and there's not much you can do with it. Other times you can fiddle with the pattern to get decimation to line up between the two sections. Other times you can blend some stuff to try and get a smooth outcome. There's no science to it you just have to mess with it and see what works. PLAIN OLD FUCKED UP SOURCE: Maybe the metrics make PG barf, maybe the source looks like barf, maybe the anime sucks and yatta is telling you that. TOO SHORT SECTIONS: If you chose to not ignore too short sections they're pretty easy to fix. Most of the time a lot of them will be grouped together so by fixing one you're fixing a chunk of them. The easiest way to fix them is probably after going to one to make sure that's the actual start of the scene (go back a frame), if so just use ctrl+up and your eyes to find out when the next real keyframe is and ctrl+q (delete section) along the way. If it's not going fast and quick and really mindnumbingly dumb you're doing it wrong. ----------------------------------- Stuff After Normal Pattern Guidance ----------------------------------- There's some general stuff you should do after pattern guidance. There may be some combing remaining because pattern guidance barfed on a scene, the last frame of a section doesn't have the right match or there may be some unnecessary post processing going on in sections that didn't get some love from pattern guidance because they were too short. After you finish pattern guidance you can finally hit save too (although I hear in recent versions of yatta the log gets saved now but whatever, you're a man and you'll do it like it's been done for generations). First thing on the agenda - VThresh Search: Make sure your settings are good (difference from both sides, 5-10 for dvd's, 30 for Transport streams) and basically hit up and fix shit as you encounter it till you can't hit up no more. Second - mmm... postprocess: Hit m and un-postprocess any frames that don't need it. Protip here: if you postprocessed an entire section back when you were doing pattern guidance skip the whole section with ctrl+up then continue mashing m. TFM is pretty damn good about what it postprocessed so don't really expect any that you didn't add yourself. Telecide was pretty retarded though in the amount of shit it post processed (especially on transport streams) but you probably didn't use Telecide for your matcher in ymc. If you're satisfied with what you've got in the right click menu you can save all overrides and everything will be generated and saved for you. Save, quit, load avs and enjoy. ------------------------------------------------------------ Pattern Guidance My Source is a Nice Transport Stream with fake soft telecine (constant pattern between commercials) ------------------------------------------------------------ If you couldn't figure this out on your own it's hopeless. Ok so it's not really soft telecine (because if it was you could just do force film in dgdecode/dvd2avi). I guess this section applies to soft telecine dvd's too if that's your thing but force film is cool too (for that). There is some debate as to whether collecting metrics really is even necessary here. I say it is because I've run into several issues described below about vthresh search that would've otherwise have gone ignored had you not spend the whole 10 minutes it takes for telecide->decimate to run. With this kind of source when you're collecting metrics you really don't need the matches from TFM (it's a waste of time) so you can just use cutter->telecide->scxvid->decimate. Actually you probably don't even need SCXvid since we'll be making our own sections anyway. Just cutter->telecide->decimate should be very quick (10 minutes on an i7). When you load it just create section start points where all the commercials breaks were and perform local pattern guidance (ctrl+alg+g) on these 4-5 sections and you're done. You're not running pattern guidance from the right click menu because it's not necessary. You still should do a vthresh search, I've had some combed frames here and there (they can be just a random frame for not apparent reason) despite the seemingly constant pattern. I've also had some here the there was one scene that something was done too which caused it to have a different pattern then the rest (but it won't error because the section we ran PG on is so large) but you'll find it with a vthresh search. Fix any other junk that needs to be fixed, filtered, whatever, and save that shit. ----------------------------------------------------------------- Pattern Guidance My Source is a Shitty Transport Stream with o9k Matching Failures (because Mentar said so) ----------------------------------------------------------------- So your source is a transport sream but it's really shitty (Zero no Tsukaima - Princess no Rondo) and pattern guidance produces over 9k matching failures. This type of pattern guidance is only meant for those types of really shitty transport streams which are hard telecine to boot. To start off with before you run pattern guidance make sure all your shit is set up and make sure the min. length for pattern guidance is 5. Second make sure your vthresh search is difference on both sides and set to 30. Whether show too short sections is on doesn't matter and you'll see why in a minute. Apply pattern guidance. Proceed to close the log (yes we're ignoring the log). Right click menu -> Recalculate Matches -> Select lowest vmetric at section bounds: Enter 50 for the threshold. This threshold is the vmetric threshold for when it gets triggered, so if the vmetric for the last frame of that section is >50 then chose the lowest vmetric out of the matches. Search for high vmetrics and fix any problems you might encounter (you'll encounter any number of problems listed in the previous pattern guidance section). The idea here is that you'll encounter any of the actual problems during the vthresh search. Do the same for postprocessed frames with M. Protip: if you don't feel like using vinverse if you put gradfunkmirror() after resizing to 1280x720 (or lower) it'll deal with any telecide combing (combing on fades) and kill two birds with one stone. Save all overrides, save, done and give praise to the Germans. --------- CHANGELOG --------- 2011-06-09: v0.41 Omit junk from prologue I didn't like 2011-06-07: v0.4 Added vfr section 2011-06-07: v0.3 Nice transport stream (constant pattern between commercials) 2011-06-07: v0.2 Mentar's shitty transport stream pattern guidance 2011-06-07: v0.12 Double for real this time, added stuff after PG 2011-06-07: v0.11 For real this time, added dumb shit I forgot 2011-06-06: v0.1 Initial creation of substandard document. ------------ CREDITS e-penor time ------------ By TheRyuu/Ryuuchin (hit me up on Rizon/Freenode/synirc) With contributions from Daemon404 And thanks to Myrsloik, Mentar, TheFluff, Emess (MisterHatt), Daemon404, Alizar, mirkosp