0. 说明

Consistency Error, 一致性误差, 特指:

PPG_want经过Transformation Model后得到Spec, Spec -> Wav, 然后用ASR实现 Wav -> MFCC -> PPG_listen, 计算KL误差
PPG_want经过Transformation Model后得到Spec, Spec -> MFCC -> PPG_listen, 计算KL误差
PPG_want经过Transformation Model后得到Mel, Mel -> MFCC -> PPG_listen, 计算KL误差

目前采用1

1. Inference & FindA & Calculate完整程序

但是由于不想搭一个既有tf又有pytorh的环境, 所以拆分为三部分

2. inference_and_findA_and_calculateCE_cbhg_Multi_part1

2.1. 输入

ppgs_inference_and_findA_and_calculateCE_list.txt

读入一个list, 包含标准ppg序列, 替换ppg序列, 中文还是英文说话人, 这些都来源于/datapool/home/hujk17/PPG_Projector; PPG直接是双语的维度

/datapool/home/hujk17/PPG_Projector/en_final_cn_log_dist2_PyTorch_findA_forChineseSpeaker_outPPGSPE_2020-11-06T10-09-46/LJ001-0092_en_ppg_original.npy|/datapool/home/hujk17/PPG_Projector/en_final_cn_log_dist2_PyTorch_findA_forChineseSpeaker_outPPGSPE_2020-11-06T10-09-46/LJ001-0092_cn_ppg_projected.npy|0
/datapool/home/hujk17/PPG_Projector/en_final_cn_log_dist2_PyTorch_findA_forChineseSpeaker_outPPGSPE_2020-11-06T10-09-46/LJ001-0092_en_ppg_original.npy|/datapool/home/hujk17/PPG_Projector/en_final_cn_log_dist2_PyTorch_findA_forChineseSpeaker_outPPGSPE_2020-11-06T10-09-46/LJ001-0092_cn_ppg_projected.npy|1
/datapool/home/hujk17/PPG_Projector/en_final_cn_log_dist2_PyTorch_findA_forChineseSpeaker_outPPGSPE_2020-11-06T10-09-46/LJ002-0019_en_ppg_original.npy|/datapool/home/hujk17/PPG_Projector/en_final_cn_log_dist2_PyTorch_findA_forChineseSpeaker_outPPGSPE_2020-11-06T10-09-46/LJ002-0019_cn_ppg_projected.npy|0
/datapool/home/hujk17/PPG_Projector/en_final_cn_log_dist2_PyTorch_findA_forChineseSpeaker_outPPGSPE_2020-11-06T10-09-46/LJ002-0019_en_ppg_original.npy|/datapool/home/hujk17/PPG_Projector/en_final_cn_log_dist2_PyTorch_findA_forChineseSpeaker_outPPGSPE_2020-11-06T10-09-46/LJ002-0019_cn_ppg_projected.npy|1

2.2. 核心代码

路径: /datapool/home/hujk17/ppg_decode_spec_10ms_sch_Multi/inference_and_findA_and_calculateCE_cbhg_Multi_part1.py

        model = DCBHG()model = tts_load(model=model, ckpt_path=ckpt_path_Multi)ppgs_list = open(ppgs_paths, 'r')ppgs_list = [i.strip() for i in ppgs_list]for idx, ppg_path_and_findA_ppg_path_and_speaker in tqdm(enumerate(ppgs_list)):ppg_path, findA_ppg_path, speaker_id = ppg_path_and_findA_ppg_path_and_speaker.split('|')ppg = np.load(ppg_path)findA_ppg = np.load(findA_ppg_path)assert ppg.shape[1] == PPG_DIM and findA_ppg.shape[1] == PPG_DIMspeaker_id = int(speaker_id)mel_pred, spec_pred, mel_pred_audio, spec_pred_audio = tts_predict(model, ppg, speaker_id)findA_mel_pred, findA_spec_pred, findA_mel_pred_audio, findA_spec_pred_audio = tts_predict(model, findA_ppg, speaker_id)# CE_fromWav, CE_seq_fromWav = consistencyError_fromWav(spec_pred_audio, ppg)# findA_CE_fromWav, findA_CE_seq_fromWav = consistencyError_fromWav(findA_spec_pred_audio, ppg)# with open(CE_fromWav_compare_path, 'w') as f:#     f.write(str(CE_fromWav) + '\n')#     f.write(str(findA_CE_fromWav) + '\n')write_wav(os.path.join(Multi_log_dir, "{}_sample_mel.wav".format(idx)), mel_pred_audio)write_wav(os.path.join(Multi_log_dir, "{}_sample_spec.wav".format(idx)), spec_pred_audio)np.save(os.path.join(Multi_log_dir, "{}_sample_mel.npy".format(idx)), mel_pred)np.save(os.path.join(Multi_log_dir, "{}_sample_spec.npy".format(idx)), spec_pred)draw_spec(os.path.join(Multi_log_dir, "{}_sample_mel.png".format(idx)), mel_pred)draw_spec(os.path.join(Multi_log_dir, "{}_sample_spec.png".format(idx)), spec_pred)write_wav(os.path.join(Multi_log_dir, "{}_sample_mel_findA.wav".format(idx)), findA_mel_pred_audio)write_wav(os.path.join(Multi_log_dir, "{}_sample_spec_findA.wav".format(idx)), findA_spec_pred_audio)np.save(os.path.join(Multi_log_dir, "{}_sample_mel_findA.npy".format(idx)), findA_mel_pred)np.save(os.path.join(Multi_log_dir, "{}_sample_spec_findA.npy".format(idx)), findA_spec_pred)draw_spec(os.path.join(Multi_log_dir, "{}_sample_mel_findA.png".format(idx)), findA_mel_pred)draw_spec(os.path.join(Multi_log_dir, "{}_sample_spec_findA.png".format(idx)), findA_spec_pred)

3. inference_and_findA_and_calculateCE_cbhg_Multi_part2_en

这个是librispeech训练出来的英文ASR里面的操作

路径: /datapool/home/hujk17/ppgs_extractor_10ms_sch_lh_librispeech/inference_and_findA_and_calculateCE_cbhg_Multi_part2_en.py

3.1. 输入

wavs_inference_and_findA_and_calculateCE_list.txt

2中合成的findA声音和不带findA的声音, 路径需要自己写入到这个list.txt中, 记住和ppgs_list.txt的顺序保持一致

3.2. 代码

略

4. inference_and_findA_and_calculateCE_cbhg_Multi_part2_cn

和3完全一样

路径: /datapool/home/hujk17/ppgs_extractor_10ms_sch_lh_aishell1/inference_and_findA_and_calculateCE_cbhg_Multi_part2_cn.py

5. inference_and_findA_and_calculateCE_cbhg_Multi_part3

compare_ppgs

list的结构, 每一行:

标准PPG
findA-PPG
标准PPG->NN->Wav->PPG_en + PPG_cn
findA-PPG->NN->Wav->PPG_en + PPG_cn
说话人0或者1(意味着跨语言或者不跨语言)

一个6个ppg路径, 被5个|隔开, 后面多一个|0或者|1

然后算距离就好了, 略

Consistency Error (一致性误差) 的计算相关推荐

PPG Project Method专利: FindA+NN+一致性误差梳理草案
0. 说明专利不同于论文, 有一些微小的区别: 老师说, "不过,实际上不用这么细,而是要把主要idea说清楚就好.具体的实现方法,不用特别详细:专利要越general.越有通用性越好.而 ...
一文读懂 Bias（偏差）、Error（误差）、Variance（方差）
一文读懂 Bias(偏差).Error(误差).Variance(方差) 偏差偏差度量了学习算法的期望预期与真实结果的偏离程度 ,即刻画了学习算法本身的拟合能力.偏差太高,就会出现欠拟合,即与真实结 ...
R语言使用ega包的plotClarkeGrid函数可视化clarke error grid误差分析图、克拉克误差网格分析：可视化误差网络图及相关拟合曲线
R语言使用ega包的plotClarkeGrid函数可视化clarke error grid误差分析图.克拉克误差网格分析:可视化误差网络图及相关拟合曲线包的安装和导入 install.packag ...
【MATLAB】进阶绘图 ( Boxplot 箱线图 | boxplot 函数 | Error Bar 误差条线图 | errorbar 函数 )
文章目录一.Boxplot 箱线图 1.boxplot 函数 2.代码示例二.Error Bar 误差条线图 1.errorbar 函数 2.代码示例一.Boxplot 箱线图 1.boxplo ...
Java黑皮书课后题第6章：**6.34（打印日历）编程练习题3.21使用Zeller一致性原理来计算某天是星期几，使用Zeller的算法简化程序清单6-12以获得每月开始的第一天是星期几
6.34(打印日历)编程练习题3.21使用Zeller一致性原理来计算某天是星期几,使用Zeller的算法简化程序清单6-12以获得每月开始的第一天是星期几题目题目描述编程练习题3.21的Zel ...
Qt MSVC 2017 32bit 编译出现“error: C2131: 表达式的计算结果不是常数”错误
在MinGW 32bit下正常编译的 qint64 fileLen = dstFile.size(); char fileBuffer[fileLen]; 切换成MSVC 2017 32bit 编译后 ...
Bias（偏差），Variance（方差），Error（误差）
一.Bias(偏差) Bias描述的是根据样本拟合出的模型的输出值与真实值之间的差距,就是模型在样本上拟合的好坏.Bias反映了模型本身的精准度. Bias越小,即模型在样本上拟合的越好.但是若要减小 ...
详解 CAP 定理 Consistency（一致性）、 Availability（可用性）、Partition tolerance（分区容错性）
详解 CAP 定理 Consistency(一致性). Availability(可用性).Partition tolerance(分区容错性) CAP原则又称CAP定理,指的是在一个分布式系统中,C ...
发票处理误差的计算方式
发现不少浏览这篇发票误差计算的文章,可能大家其实不是想要之前的那种,之前只是记录一个发票整单税额误差校验计算的正确方式,因为这个校验比较容易出现误区,所以只是纯粹的记录下一个数值用例,但不懂业务的人应 ...

Consistency Error (一致性误差) 的计算