[Tool] Sync subtitles with XCorSync

Discuss fansubbing issues, problems, or learn how to create fansubs.
User avatar
RetroHelix
Posts: 435
Joined: Sat Mar 14, 2009 4:47 pm
Location: Germany

[Tool] Sync subtitles with XCorSync

Postby RetroHelix » Sun Feb 15, 2015 4:09 pm

XCorSync

I wrote XCorSync to make the task of synchronizing subtitles to different videos more convenient. The program puts out an offset that shows you the delay between two video sources. You can even sync every line of a subtitle file automatically. I hope this will be usefull for the transcribing project by Avallac'h.

Image

How it works:
XCorSync will check if two videos are in sync by comparing their audio streams. It extracts the audio streams using ffmpeg and cuts them recoding to the selected line of your subtitle.
For both audio snippets the correlation coefficient of their audio fingerprints (Generated by fpcalc) gets calculated and compared. When the correlation coefficient is under a given threshold the audio snippets aren't in sync and the time offset between both gets calculated by using cross correlation (praat will do this job). To display and work with subtitle files the Subtitle Edit Light Library is used.
In short: Load the video files and the subtitle file, select and right click into the subtitle view and select an option to see how much the two audio streams differ. You can even start an automatic mode that tries to sync every line from your subtitle file (this takes about 2.5 - 3 hours for 50 minutes using the default settings!)

Settings and functions:
Since XCorSync is more a GUI for the different tools (See the Apps directory) that are used to extract, convert and calculate the offset between two video files than a standalone program, a console window gets opened every time the functionality of another tool is needed. If the console window pops up and ask for permission to overwrite a file enter "y" for yes and press enter so that a new file gets written. This also means you can abort converting if you work with the same files again.
XCorSync will extract/convert the audio streams of the video files you input in the Files section. This can take some time so be patient. After loading a subtitle file you can select a line. By right clicking into the subtitle section you can choose between four options:
"Copy to clipboard" to copy the content of the selected line to the clipboard.
"Check if sync" to compare the audio fingerprints and determine if the selected line is in sync. The correlation threshold is used to determine the outcome. Two signals are sync if the correlation coefficient is 1 but since two audiofiles can differ slightly the default value is set to 0.6.
"Synchronize (compare fingerprints first)" will check if the line is sync and will only do a cross correlation to calculate the time delay if the input files are not in sync.
"Synchronize (without comparing fingerprints)" will start the cross correlation right away without checking and outputs an offset.
The "Sync all lines" button will start a lengthy process that calculates the time delay for every line in your subtitle file.
Since the audiofingerprinting technology is not meant for small snippets of some seconds a "Fingerprint length" can be entered. The value entered there sets the length for the audio snippet that is getting cut from the whole audiostream.
The "X-Correlation Range" lets you change the range in which the cross correlation is used. When using cross correlation you basically just compare two signals by calculating their correlation coefficient while shifting one signal and calculating the correlation coefficient again. The output for the time offset is where the correlation coefficient has it's maximum (here the two signals are "most similar"). The default value of 30 seconds must be changed if there is a new scene that is longer than 30 seconds in your new video file. I compared a TV-Rip with a DVDRip and there were 322 seconds of new material but no additional scene was longer than 30 seconds so this value should be ok.

Bugs and sanity checks:
There are most likely bugs in this program and I don't bothered to implement some rudimentary checks. The program will crash if you haven't selected an input file and choose to synchronize it. Same goes if you open unsupported files (documents instead of videos or subtitle files or who knows what...).
If there are some critical bugs then just write me a PM or reply in this thread and I will see what I can do.

Sourcecode:
I'm not good at programming but I know how to use C# to get things done. PM me for the sourcecode of the program but be warned I not really made use of the possibilities of OOP. I just pasted everything together :D
I would also be really happy if someone implements cross correlation into subedit or another program. Maybe as a plugin.

Links of interest (and of the tools I used):
https://acoustid.org/chromaprint for the fpcalc tool.
https://www.ffmpeg.org used to extract/cut/convert files.
http://www.praat.org/ used for the cross correlation.
https://code.google.com/p/subtitleedit/ library used to load the subtitle files.
http://www.randombytes.org/audio_comparison.html the site that gave me the idea for this tool.

Greetings to:
Avallac'h, saigo_x, xaxa, Keiko1981, ZemusDS, k361 and everyone else who works on the transcribing project by Avallac'h.
Chuks and the guys from Ame No Uta for some wonderful oldies.
And to everyone else who uploads or subs doramas.

Download: https://mega.co.nz/#!GAoFAKgb!dXx0K_zle ... 4_tvLO_ygg
Subtitle Edit Plugin: https://mega.co.nz/#!mBRWVIaA!zw6Ts2OrW ... kPUynFW6XU (extract into plugin folder of Subtitle Edit)
Last edited by RetroHelix on Sun May 24, 2015 5:28 pm, edited 7 times in total.

User avatar
RetroHelix
Posts: 435
Joined: Sat Mar 14, 2009 4:47 pm
Location: Germany

Re: [Tool] Sync subtitles with XCorSync

Postby RetroHelix » Sun Mar 01, 2015 1:08 pm

I tested the tool some more and came to the decision that I have to correct some things to make the tool easier to use and more correct in regards to the offset it finds out. Just wanted to post here in case somebody noticed that the tool is not working right.

EDIT: I updated the program. Many fixes and an automatic mode were added: see first post for link.
Last edited by RetroHelix on Sun May 24, 2015 5:29 pm, edited 1 time in total.

User avatar
RetroHelix
Posts: 435
Joined: Sat Mar 14, 2009 4:47 pm
Location: Germany

Re: [Tool] Sync subtitles with XCorSync

Postby RetroHelix » Sun May 24, 2015 5:28 pm

I made a plugin for Subtitle Edit and fixed a bug.

Image

You can open the plugin in Subtitle Edit by choosing "Use cross correlation to sync subtitles" in the synchronization menu. By selecting a line and choosing synchronize the program will calculate an offset. You can use this offset to correct your subs with Subtitle Edit.

Downloadlink in first post.

civil78
Posts: 1
Joined: Mon Sep 12, 2016 8:09 am

Re: [Tool] Sync subtitles with XCorSync

Postby civil78 » Mon Sep 12, 2016 8:57 am

Nice app, as you said in your post can you send me the source to check if it possible to add a job queue feature and to improve speed with multi threading option (if your algorithm support this).

Thanks.

User avatar
RetroHelix
Posts: 435
Joined: Sat Mar 14, 2009 4:47 pm
Location: Germany

Re: [Tool] Sync subtitles with XCorSync

Postby RetroHelix » Mon Sep 12, 2016 6:49 pm

civil78 wrote:Nice app, as you said in your post can you send me the source to check if it possible to add a job queue feature and to improve speed with multi threading option (if your algorithm support this).
Thanks.

Nice to see someone using this :)
Here is the source: Link. Unfortunately I don't have the sourcefiles for the plugin version anymore. And as I said before this really is some dirty, copypaste and forget code :D
If you really want to speed up everything I would suggest you to write your own tool using https://github.com/protyposis/Aurio / https://github.com/protyposis/AudioAlign. It allows using audio fingerprinting (subfringerprints) for comparison of audio files and supports a variety of different matching algorithms. Just drag two audio files into AudioAlign and press Match&Align -> Start to see its speed (looks like 1000% faster than just crosscorelating).
If anything comes from this please keep me updated.

Good luck.

JackBauer2424
Posts: 169
Joined: Mon May 11, 2015 4:48 am

Re: [Tool] Sync subtitles with XCorSync

Postby JackBauer2424 » Wed Sep 21, 2016 8:25 am

I'm trying to watch Eigyo Bucho Kira Natsuko with Nanako, and episodes 5 and 7 and possibly others have subtitles so out of sync it's unwatchable. It's so weird reading subs 20 seconds or so ahead of the actual video, especially if you understand some Japanese. The subtitles kept going even during the opening credits where there was only sponsors being spoken.


Return to “Fansubbing”

Who is online

Users browsing this forum: No registered users and 1 guest