Re: "Extract LTC from audio and align video" menu option missing from "Transcode/Import Video file" popup
robertlazarski <[email protected]>
| Newsgroups | gmane.comp.audio.ardour.user |
|---|---|
| Message-ID | <CABpPLBUxNQ6Z74QAAJNJaZyq5pWhBB0xXb86RbjRUQi4rgAGcg@mail.gmail.com> |
Just to close this thread out, I finally was able to write a script that combines two camera videos. I do have one problem still with Ardour or ffmpeg, please see my update in-line. On Tue, Apr 24, 2018 at 1:30 PM, robertlazarski <[email protected]> wrote: > I have another problem now with this phone, I can't get the phone video > wav file sample position per LTC frame to match the q8 sample position. > This is what I think is causing a 2 second delay sync problem in a 3 minute > or so in a test video that has parts from both the q8 and the phone. > > > I can't seem to fix this, each LTC frame has a different size in the phone > wav file compared to the q8.wav file. Both of the original videos show > "CABAC / 1 Ref Frames" . However the phone video audio was recorded as AAC > while the q8 video audio was recorded as PCM so maybe that is the issue. > > # phone video in mp4 format and AAC audio, cannot fix this with ffmpeg by > any conversion I tried > 00000000 00:26:55:01 | 3118 4715 # (4715 - 3118) = 1597 > > # q8 mov video and PCM audio > 00000000 00:26:55:01 | 3123 4724 # (4724 - 3132) = 1601 > > Any idea how to fix this? > > > There was several issues here, I suspect write speed was one of them so I put in class 10 SD in the Q8 pair and F8 which seemed to help. Really the main issue though was scripting ffmpeg with segments didn't pan out. I asked a question on the ffmpeg list but had no response, I will follow up there with the details. I had much better luck using ffmpeg "ss" and "to" params by calculating the times with the sample position of the ltc. I was able to do a calculated camera transition between the pair of Q8 cameras that way with a little less than .5 millseconds or around 20 sample positions missing in the transitions. I tried to do better but ran out of ideas. I may try again at some point. The one Ardour problem I still have is that when exporting a video, I do not see the PMC option for the F8 24 bit audio, I only see pcm_s16le . See the output below from mediainfo on the F8 file: Audio Format : PCM Format settings : Little / Signed Codec ID : 1 Duration : 3 min 10 s Bit rate mode : Constant Bit rate : 4 608 kb/s Channel(s) : 4 channels Sampling rate : 48.0 kHz Bit depth : 24 bits Stream size : 104 MiB (100%) I see this from ffmpeg, I am not sure if I need 24 bits or 32 bits? ffmpeg -v error -formats | grep -i pcm | grep 24 | grep little-endian | grep -v unsigned DE s24le PCM signed 24-bit little-endian ffmpeg -v error -formats | grep -i pcm | grep 32 | grep little-endian | grep -v unsigned DE f32le PCM 32-bit floating-point little-endian DE s32le PCM signed 32-bit little-endian However in Ardour when exporting a mov video I only see this pcm audio option in the audio codec drop down: pcm_s16le Any guidance appreciated. This is how my script turned out. At some point I may install Davinci Resolve but it requires CentOS 7 and that's not my distro. This script will be what I use for the next few months as I kind of like the idea of auto editing. Will see how far it gets me. #!/bin/bash # This script generates a single video from parts of videos from 2 Zoom Q8 cameras # by using ltc on ch1 of each video. The priority is as little ltc audio drift as possible, # script performance and extra transcoding are not an issue. Little audio drift in this case # means the start and end times match the source videos closely, with no dropped video frames # nor dropped ltc frames. I was able to get within .2 seconds by the end of the video, # and while I have no immediate ideas how to do better I will continue to try. # The ltc comes from 2 Tentacle Sync units, that were 'jammed' to a # Zoom F8. The main idea is to sync the video with the BWF 8 track audio generated on # on the Zoom F8 via Ardour. This is possible by using the BWF timecode metadata, and # ltc on ch1 of the video. ch2 contains a scratch track. # A fixed duration such as 20 seconds is set, indicating a camera # transision. Videos are mov at 1280x720 with ltc audio at 48KHZ / 16 bits. # usage: # /home/myuser/input> ls # output parseLTC.sh q81.mov q82.mov # /home/myuser/input> cd output/ # /home/myuser/input/output> sh ../parseLTC.sh # clear any files from previous processing rm -f *.txt rm -f *cut* rm -f *.csv # read LTC from wav files, first camera is q81 and the second camera is q82 ffmpeg -y -v error -nostdin -i ../q81.mov -vn -c:a pcm_s16le -ar 48000 -ac 2 q81.wav ffmpeg -y -v error -nostdin -i ../q82.mov -vn -c:a pcm_s16le -ar 48000 -ac 2 q82.wav ltcdump -f 29.97 q81.wav 2> /dev/null > q81.ltc.txt ltcdump -f 29.97 q82.wav 2> /dev/null > q82.ltc.txt # some errors I found while playing option bingo with ffmpeg validate() { lastTimecodeInSegment=0 lastTimecodeInSegment=`tail -n 1 ltc.txt | awk '{ print $2 }' | sed 's/\(.*\):/\1./' ` # can sometimes receive unparsable dates if [[ ${#lastTimecodeInSegment} -ne 11 ]]; then lastTimecodeInSegment=0 printf "\nError processing file %s\n" "${fileToProcess}" printf "\ntimecode number of chars is not correct: " printf " %s" "${#lastTimecodeInSegment}" return fi # can receive invalid time such as: 00000680 00:13:61:10 | 22 1609 if ! date -d "$lastTimecodeInSegment" >/dev/null 2>&1; then printf "\nError processing file %s\n" "${fileToProcess}" printf "\ninvalid date format:" printf " %s" "$lastTimecodeInSegment" return fi epoch_secs_ltc_to_process=$(date +%s --date="$timecode_date ${lastTimecodeInSegment}") ltc_to_process_formatted="$timecode_date ${lastTimecodeInSegment}" printf "\nltc_to_process_formatted: $ltc_to_process_formatted" if [[ $epoch_secs_ltc_to_process -ne 0 && $epoch_secs_stop_time -ne 0 && $epoch_secs_ltc_to_process -ge $epoch_secs_stop_time ]]; then found_stop_time=1 printf "\nskipping file %s\n" "${fileToProcess}" printf "\ntimecode found:" printf " %s" "${ltc_to_process_formatted}" printf "\nis after or equal to stop time: " printf " %s" "${epoch_secs_stop_time_formatted}" return fi startingSamplePositionInSegment=0 startingSamplePositionInSegment=`tail -n 1 ltc.txt | awk '{ print $4 }' | sed 's/\(.*\):/\1./' ` endingSamplePositionInSegment=0 endingSamplePositionInSegment=`tail -n 1 ltc.txt | awk '{ print $5 }' | sed 's/\(.*\):/\1./' ` printf "\nltc seems valid" isLTCDumpOutputValid=1 } # define timecode start time, just add date timecode_date=2018-05-16 # skip segments before this timecode epoch_secs_start_time=$(date +%s --date="$timecode_date 00:04:25") start_timecode=`date +%H:%M:%S:%N --date="Jan 1, 1970 00:00:00 +0000 $epoch_secs_start_time seconds" | cut -b1-11 | sed 's/\(.*\):/\1./' ` start_time_formatted="$timecode_date $start_timecode" # define timecode stop time epoch_secs_stop_time=$(date +%s --date="$timecode_date 00:07:10") stop_timecode=`date +%H:%M:%S:%N --date="Jan 1, 1970 00:00:00 +0000 $epoch_secs_stop_time seconds" | cut -b1-11 | sed 's/\(.*\):/\1./' ` stop_time_formatted="$timecode_date ${stop_timecode}" timeDivisionInSeconds=20 let epoch_secs_timecode_to_find=$((epoch_secs_start_time + timeDivisionInSeconds)) find_timecode=`date +%H:%M:%S:%N --date="Jan 1, 1970 00:00:00 +0000 $epoch_secs_timecode_to_find seconds" | cut -b1-11 | sed 's/\(.*\):/\1./' ` find_timecode_formatted="$timecode_date ${find_timecode}" let epoch_secs_next_timecode_to_find=$epoch_secs_timecode_to_find+$timeDivisionInSeconds next_timecode_to_find=`date +%H:%M:%S:%N --date="Jan 1, 1970 00:00:00 +0000 $epoch_secs_next_timecode_to_find seconds" | cut -b1-11 | sed 's/\(.*\):/\1./' ` next_timecode_to_find_formatted="$timecode_date ${next_timecode_to_find}" q81HasLTCStartTime=0 hasRejectBeforeTime=0 q81TimeCodeInProgress=0 q82TimeCodeInProgress=0 q81PositionTimeStart='' q81PositionTimeStop='' q82PositionTimeStart='' q82PositionTimeStop='' fourtyEightKHZ=48000 while read -r lineq81 do ltc_line='' ltc_line="$lineq81" hasLTC=0 hasLTC=`echo $ltc_line | grep -v '#' | grep ':' | wc -w` if [[ $hasLTC -eq 0 ]]; then printf "\ncannot parse timecode on line: $ltc_line \n" continue fi rm -f ltc.txt # find timecode in format 'HH:MM:SS.00' with frames as last 2 digits used for .00 time. # remove occasional '.' in the form 00:00:00.0 instead of 00:00:00:0 # remove negative values # disallow 0000000f 00:12:25:45 echo $ltc_line | grep -v '#' | grep 00000000 | grep -v '\.' | grep -v '-' > ltc.txt validate if [[ $found_stop_time -eq 1 ]]; then printf "\nq81 video is after stop time, executing break after writing last csv entry\n" if [[ $q81PositionTimeStart != '' && $q81PositionTimeStop != '' ]]; then printf "\nExecuting: echo $q81PositionTimeStart","$q81PositionTimeStop >> q81_out.csv" echo $q81PositionTimeStart","$q81PositionTimeStop >> q81_out.csv fi break fi if [[ $q81HasLTCStartTime -eq 0 && $isLTCDumpOutputValid -eq 0 ]]; then printf "\non q81HasLTCStartTime=0 and isLTCDumpOutputValid=0 ... skipping" continue fi if [[ $q81HasLTCStartTime -eq 0 && $isLTCDumpOutputValid -eq 1 ]]; then dateDiff=0 dateDiff=`date -d "$ltc_to_process_formatted $(date -d "$start_time_formatted" +%s.%N) seconds ago" +%s.%N` if awk 'BEGIN{exit ARGV[1]>ARGV[2]}' "0" "$dateDiff" then q81HasLTCStartTime=1 printf "\non dateDiff $dateDiff , ltc_to_process_formatted $ltc_to_process_formatted is later than or equal to start_time_formatted $start_time_formatted" printf "\nproceeding ...\n" samplePositionOffsetQ81=0 samplePositionOffsetQ81=$(awk "BEGIN {printf \"%.6f\",${startingSamplePositionInSegment}/${fourtyEightKHZ}}") q81PositionTimeStart=$samplePositionOffsetQ81 printf "\ncalculated first start position on q81: $q81PositionTimeStart\n" else printf "\non dateDiff $dateDiff , ltc_to_process_formatted $ltc_to_process_formatted is before start_time_formatted $start_time_formatted" printf "\nskipping ...\n" continue fi fi if [[ $isLTCDumpOutputValid -eq 0 && $q81TimeCodeInProgress -eq 1 ]]; then printf "\nq81 video is the default and is in progress, occasional bad timecode values are ok" continue fi if [[ $isLTCDumpOutputValid -eq 0 && $q81TimeCodeInProgress -eq 0 ]]; then printf "\nskipping , ltc is invalid and q81TimeCodeInProgress is false: " continue fi if [[ $hasRejectBeforeTime -eq 1 ]]; then dateDiff=0 dateDiff=`date -d "$ltc_to_process_formatted $(date -d "$rejectBeforeTime_formatted" +%s.%N) seconds ago" +%s.%N` if awk 'BEGIN{exit ARGV[1]>ARGV[2]}' "$dateDiff" "0" then printf "\nexecuting continue , dateDiff $dateDiff , ltc_to_process_formatted $ltc_to_process_formatted is before or equal to rejectBeforeTime_formatted $rejectBeforeTime_formatted \n" continue else printf "\nproceeding , dateDiff $dateDiff , ltc_to_process_formatted $ltc_to_process_formatted is after $rejectBeforeTime_formatted \n" fi fi dateDiff=0 dateDiff=`date -d "$ltc_to_process_formatted $(date -d "$find_timecode_formatted" +%s.%N) seconds ago" +%s.%N` if awk 'BEGIN{exit ARGV[1]>ARGV[2]}' "0" "$dateDiff" then printf "\non dateDiff $dateDiff , ltc_to_process_formatted $ltc_to_process_formatted is later than or equal to find_timecode_formatted $find_timecode_formatted \n" else printf "\non dateDiff $dateDiff , ltc_to_process_formatted $ltc_to_process_formatted is less than find_timecode_formatted $find_timecode_formatted \n" printf "\nproceeding ..." if [[ $q81TimeCodeInProgress -eq 0 ]]; then samplePositionOffsetQ81=0 samplePositionOffsetQ81=$(awk "BEGIN {printf \"%.6f\",${startingSamplePositionInSegment}/${fourtyEightKHZ}}") q81PositionTimeStart=$samplePositionOffsetQ81 printf "\ncalculated samplePositionOffsetQ81: $samplePositionOffsetQ81 , from $startingSamplePositionInSegment / $fourtyEightKHZ" else # will reset on each loop q81PositionTimeStop=0 q81PositionTimeStop=$(awk "BEGIN {printf \"%.6f\",${endingSamplePositionInSegment}/${fourtyEightKHZ}}") printf "\nfound q81PositionTimeStop $q81PositionTimeStop" fi q81TimeCodeInProgress=1 continue fi printf "\nfound find_timecode: " printf " %s" "${find_timecode}" printf "\nwill now process the q82 files looking for timecode later than or equal to: " printf " %s" "${find_timecode_formatted}" printf "\nand timecode before or equal to: " printf " %s" "${next_timecode_to_find_formatted}" if [[ $q81PositionTimeStart != '' && $q81PositionTimeStop != '' ]]; then printf "\nExecuting: echo $q81PositionTimeStart","$q81PositionTimeStop >> q81_out.csv" echo $q81PositionTimeStart","$q81PositionTimeStop >> q81_out.csv else printf "\nloop completed without valid timecode, executing break" break fi q81PositionTimeStop=0 q81TimeCodeInProgress=0 samplePositionOffsetQ82=0 q81PositionTimeStart='' q81PositionTimeStop='' while read -r lineq82 do ltc_line='' ltc_line="$lineq82" hasLTC=0 hasLTC=`echo $ltc_line | grep -v '#' | grep ':' | wc -w` if [[ $hasLTC -eq 0 ]]; then printf "\ncannot parse timecode on line: $ltc_line \n" continue fi rm -f ltc.txt # find timecode in format 'HH:MM:SS.00' with frames as last 2 digits used for .00 time. # remove occasional '.' in the form 00:00:00.0 instead of 00:00:00:0 # remove negative values # disallow 0000000f 00:12:25:45 echo $ltc_line | grep -v '#' | grep 00000000 | grep -v '\.' | grep -v '-' > ltc.txt validate if [[ $found_stop_time -eq 1 ]]; then printf "\nq82 video is after stop time, executing break after writing last csv entry\n" break fi if [[ $isLTCDumpOutputValid -eq 0 && $q82TimeCodeInProgress -eq 1 ]]; then printf "\nfound LTC error while q82TimeCodeInProgress=1, skipping" continue fi if [[ $isLTCDumpOutputValid -eq 0 ]]; then printf "\nFound ltc error outside time range while processing q82 , skipping" continue fi dateDiff=0 dateDiff=`date -d "$ltc_to_process_formatted $(date -d "$find_timecode_formatted" +%s.%N) seconds ago" +%s.%N` if awk 'BEGIN{exit ARGV[1]>ARGV[2]}' "0" "$dateDiff" then printf "\nproceeding , dateDiff $dateDiff , ltc_to_process_formatted $ltc_to_process_formatted is later or equal to find_timecode_formatted $find_timecode_formatted" else printf "\nskipping q82 timecode on dateDiff $dateDiff , ltc_to_process_formatted $ltc_to_process_formatted is before find_timecode_formatted $find_timecode_formatted" continue fi dateDiff=0 dateDiff=`date -d "$ltc_to_process_formatted $(date -d "$next_timecode_to_find_formatted" +%s.%N) seconds ago" +%s.%N` if awk 'BEGIN{exit ARGV[1]>ARGV[2]}' "$dateDiff" "0" then printf "\nproceeding on dateDiff $dateDiff , $ltc_to_process_formatted is before or equal to $next_timecode_to_find_formatted" else printf "\nexecuting break , dateDiff $dateDiff , ltc_to_process_formatted $ltc_to_process_formatted is greater than next_timecode_to_find $next_timecode_to_find" break fi if [[ $q82TimeCodeInProgress -eq 0 ]]; then samplePositionOffsetQ82=0 samplePositionOffsetQ82=$(awk "BEGIN {printf \"%.6f\",${startingSamplePositionInSegment}/${fourtyEightKHZ}}") q82PositionTimeStart=$samplePositionOffsetQ82 printf "\ncalculated q82PositionTimeStart: $q82PositionTimeStart , from $startingSamplePositionInSegment / $fourtyEightKHZ" else # will reset on each loop samplePositionOffsetQ82=0 samplePositionOffsetQ82=$(awk "BEGIN {printf \"%.6f\",${endingSamplePositionInSegment}/${fourtyEightKHZ}}") q82PositionTimeStop=$samplePositionOffsetQ82 printf "\ncalculated q82PositionStopTime: $q82PositionTimeStop , from $endingSamplePositionInSegment / $fourtyEightKHZ" fi q82TimeCodeInProgress=1 printf "\nfound ltc in range. loop ended" done < q82.ltc.txt if [[ $q82PositionTimeStart != '' && $q82PositionTimeStop != '' ]]; then printf "\nExecuting: echo $q82PositionTimeStart","$q82PositionTimeStop >> q82_out.csv" echo $q82PositionTimeStart","$q82PositionTimeStop >> q82_out.csv else printf "\nloop completed without valid timecode, executing break" break fi q82TimeCodeInProgress=0 q82PositionTimeStart='' q82PositionTimeStop='' # reset everything for next loop let epoch_secs_rejectBeforeTime=$epoch_secs_next_timecode_to_find reject_timecode=`date +%H:%M:%S:%N --date="Jan 1, 1970 00:00:00 +0000 $epoch_secs_rejectBeforeTime seconds" | cut -b1-11 | sed 's/\(.*\):/\1./' ` rejectBeforeTime_formatted="$timecode_date ${reject_timecode}" hasRejectBeforeTime=1 increment=$(( $timeDivisionInSeconds * 2 )) let epoch_secs_timecode_to_find=$epoch_secs_timecode_to_find+$increment find_timecode=`date +%H:%M:%S:%N --date="Jan 1, 1970 00:00:00 +0000 $epoch_secs_timecode_to_find seconds" | cut -b1-11 | sed 's/\(.*\):/\1./' ` find_timecode_formatted="$timecode_date ${find_timecode}" let epoch_secs_next_timecode_to_find=$epoch_secs_next_timecode_to_find+$increment next_timecode_to_find=`date +%H:%M:%S:%N --date="Jan 1, 1970 00:00:00 +0000 $epoch_secs_next_timecode_to_find seconds" | cut -b1-11 | sed 's/\(.*\):/\1./' ` next_timecode_to_find_formatted="$timecode_date ${next_timecode_to_find}" printf "\nloop completed, timecode_to_find has been reset: " printf " %s" "${find_timecode_formatted}" printf "\nnext_timecode_to_find_formatted has been reset: " printf " %s" "${next_timecode_to_find_formatted}" printf "\nrejectBeforeTime_formatted has been reset: " printf " %s" "${rejectBeforeTime_formatted}" done < q81.ltc.txt xx=0 while IFS=, read -r col1 col2 do printf "\nExecuting: ffmpeg -y -v error -nostdin -i ../q81.mov -ss $col1 -to $col2 -c:a pcm_s16le -c:v libx264 -refs 1 -x264opts b-pyramid=0 -r 30000/1001 -threads 0 -s 1280x720 -b:v: 1024k -bufsize 1216k -maxrate 1280k -b:a 192k -sample_fmt s16 -ac 2 -ar 48000 -af "aresample=async=1:min_hard_comp=0.000100:first_pts=0" -crf 0 q81_cut_"$xx".mov" ffmpeg -y -v error -nostdin -i ../q81.mov -ss $col1 -to $col2 -c:a pcm_s16le -c:v libx264 -refs 1 -x264opts b-pyramid=0 -r 30000/1001 -threads 0 -s 1280x720 -b:v: 1024k -bufsize 1216k -maxrate 1280k -b:a 192k -sample_fmt s16 -ac 2 -ar 48000 -af "aresample=async=1:min_hard_comp=0.000100:first_pts=0" -crf 0 q81_cut_"$xx".mov # debug printf "\nExecuting: ffmpeg -y -v error -nostdin -i q81_cut_"$xx".mov -vn -acodec pcm_s16le -ar 48000 -ac 2 "q81_cut_"$xx".mov.wav" " ffmpeg -y -v error -nostdin -i q81_cut_"$xx".mov -vn -acodec pcm_s16le -ar 48000 -ac 2 "q81_cut_"$xx".mov.wav" xx=$((xx + 1)) done < q81_out.csv yy=0 while IFS=, read -r col1 col2 do printf "\nExecuting: ffmpeg -y -v error -nostdin -i ../q82.mov -ss $col1 -to $col2 -c:a pcm_s16le -c:v libx264 -refs 1 -x264opts b-pyramid=0 -r 30000/1001 -threads 0 -s 1280x720 -b:v: 1024k -bufsize 1216k -maxrate 1280k -b:a 192k -sample_fmt s16 -ac 2 -ar 48000 -af "aresample=async=1:min_hard_comp=0.000100:first_pts=0" -crf 0 q82_cut_"$yy".mov" ffmpeg -y -v error -nostdin -i ../q82.mov -ss $col1 -to $col2 -c:a pcm_s16le -c:v libx264 -refs 1 -x264opts b-pyramid=0 -r 30000/1001 -threads 0 -s 1280x720 -b:v: 1024k -bufsize 1216k -maxrate 1280k -b:a 192k -sample_fmt s16 -ac 2 -ar 48000 -af "aresample=async=1:min_hard_comp=0.000100:first_pts=0" -crf 0 q82_cut_"$yy".mov printf "\nExecuting: ffmpeg -y -v error -nostdin -i q82_cut_"$yy".mov -vn -acodec pcm_s16le -ar 48000 -ac 2 "q82_cut_"$yy".mov.wav" " # debug ffmpeg -y -v error -nostdin -i q82_cut_"$yy".mov -vn -acodec pcm_s16le -ar 48000 -ac 2 "q82_cut_"$yy".mov.wav" yy=$((yy + 1)) done < q82_out.csv zz=0 if [[ $xx -eq $yy ]]; then zz=$xx elif [[ $xx -gt $yy ]]; then zz=$xx else zz=$yy fi for (( c=0; c<$zz; c++ )) do printf "\nExecuting: ls *cut_"$c"*.mov | sort | sed 's/^/file /' >> cuts.txt" ls *cut_"$c"*.mov | sort | sed 's/^/file /' >> cuts.txt done printf "\nExecuting: ffmpeg -y -v error -nostdin -f concat -fflags +genpts -safe 0 -i cuts.txt -c:a pcm_s16le -c:v libx264 -refs 1 -x264opts b-pyramid=0 -r 30000/1001 -threads 0 -s 1280x720 -b:v: 1024k -bufsize 1216k -maxrate 1280k -b:a 192k -sample_fmt s16 -ac 2 -ar 48000 -af "aresample=async=1:min_hard_comp=0.000100:first_pts=0" -crf 0 cuts.mov" ffmpeg -y -v error -nostdin -f concat -fflags +genpts -safe 0 -i cuts.txt -c:a pcm_s16le -c:v libx264 -refs 1 -x264opts b-pyramid=0 -r 30000/1001 -threads 0 -s 1280x720 -b:v: 1024k -bufsize 1216k -maxrate 1280k -b:a 192k -sample_fmt s16 -ac 2 -ar 48000 -af "aresample=async=1:min_hard_comp=0.000100:first_pts=0" -crf 0 cuts.mov # debug ffmpeg -y -v error -nostdin -i cuts.mov -vn -c:a pcm_s16le -ar 48000 -ac 2 cuts.wav exit _______________________________________________ Ardour-Users mailing list [email protected] To unsubscribe or change your mailing preferences please visit: http://lists.ardour.org/listinfo.cgi/ardour-users-ardour.org