Recurrent-VLN-BERT

Recurrent VLN-BERT复现遇到的bug

create docker

sudo nvidia-docker run -it --mount type=bind,source=/home/oem/Desktop/Zeyue/Attack/Matterport3DSimulator,target=/home/oem/Desktop/Zeyue/Attack/Matterport3DSimulator --volume `pwd`:/root/mount/Matterport3DSimulator mattersim:9.2-devel-ubuntu18.04

BUG:

    return [get_point_angle_feature(baseViewId) for baseViewId in range(36)]File "/home/oem/Desktop/Zeyue/Attack/Matterport3DSimulator/VLN/Recurrent-VLN-BERT/r2r_src/utils.py", line 378, in <listcomp>return [get_point_angle_feature(baseViewId) for baseViewId in range(36)]File "/home/oem/Desktop/Zeyue/Attack/Matterport3DSimulator/VLN/Recurrent-VLN-BERT/r2r_src/utils.py", line 363, in get_point_angle_featuresim.newEpisode('ZMojNkEp431', '2f4d90acd4024c269fb0efe49a8ac540', 0, math.radians(-30))
TypeError: newEpisode(): incompatible function arguments. The following argument types are supported:1. (self: MatterSim.Simulator, arg0: List[str], arg1: List[str], arg2: List[float], arg3: List[float]) -> NoneInvoked with: <MatterSim.Simulator object at 0x7fee5ec38180>, 'ZMojNkEp431', '2f4d90acd4024c269fb0efe49a8ac540', 0, -0.5235987755982988

sim = new_simulator()feature = np.empty((36, args.angle_feat_size), np.float32)
base_heading = (baseViewId % 12) * math.radians(30)
for ix in range(36):if ix == 0:sim.newEpisode(['ZMojNkEp431'], ['2f4d90acd4024c269fb0efe49a8ac540'], [0], [math.radians(-30)])elif ix % 12 == 0:sim.makeAction([0], [1.0], [1.0])else:sim.makeAction([0], [1.0], [0])state = sim.getState()[0]# print(state,ix)assert state.viewIndex == ix

解决方法：按格式改

AttributeError: 'list' object has no attribute 'viewIndex'

解决方法：state = sim.getState()改为state = sim.getState()[0]

ModuleNotFoundError: No module named 'transformers.pytorch_transformers'

解决方法： pytorch_transformers去掉

OSError: file Prevalent/pretrained_model/pytorch_model.bin not found

原因: 没有把预训练权重导进去
解决方法：把相应的地址改一下:
/home/tianzeyue/EQA_attack/Recurrent-VLN-BERT/weight/pytorch_model.bin

TypeError: init_weights() takes 1 positional argument but 2 were given

解决方法：
self.apply(self.init_weights)改为self.init_weights()
参考: https://github.com/lonePatient/Bert-Multi-Label-Text-Classification/issues/24

  File "r2r_src/train.py", line 198, in train_valtrain_env = R2RBatch(feat_dict, batch_size=args.batchSize, splits=['train'], tokenizer=tok)File "/home/oem/Desktop/Zeyue/Attack/Matterport3DSimulator/VLN/Recurrent-VLN-BERT/r2r_src/env.py", line 96, in __init__self.env = EnvBatch(feature_store=feature_store, batch_size=batch_size)File "/home/oem/Desktop/Zeyue/Attack/Matterport3DSimulator/VLN/Recurrent-VLN-BERT/r2r_src/env.py", line 55, in __init__sim.init()
AttributeError: 'MatterSim.Simulator' object has no attribute 'init'

解决方法：
sim.init()改为sim.initialize()
作者提到是版本原因: https://github.com/YicongHong/Recurrent-VLN-BERT/issues/4

    param.data = fn(param.data)File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 265, in <lambda>return self._apply(lambda t: t.cuda(device))File "/usr/local/lib/python3.6/dist-packages/torch/cuda/__init__.py", line 163, in _lazy_inittorch._C._cuda_init()
RuntimeError: cuda runtime error (38) : no CUDA-capable device is detected at /pytorch/aten/src/THC/THCGeneral.cpp:51

single GPU
解决方法： os.environ[‘CUDA_VISIBLE_DEVICES’] = ‘0’

  File "/home/oem/Desktop/Zeyue/Attack/Matterport3DSimulator/VLN/Recurrent-VLN-BERT/r2r_src/agent.py", line 571, in loadstates = torch.load(path)File "/usr/local/lib/python3.6/dist-packages/torch/serialization.py", line 387, in loadreturn _load(f, map_location, pickle_module, **pickle_load_args)File "/usr/local/lib/python3.6/dist-packages/torch/serialization.py", line 560, in _loadraise RuntimeError("{} is a zip archive (did you mean to use torch.jit.load()?)".format(f.name))
RuntimeError: snap/VLNBERT-PREVALENT-final/state_dict/best_val_unseen is a zip archive (did you mean to use torch.jit.load()?)

解决方法： torch.jit.load()
update the pytorch version

  File "/home/oem/Desktop/Zeyue/Attack/Matterport3DSimulator/VLN/Recurrent-VLN-BERT/r2r_src/vlnbert/vlnbert_PREVALENT.py", line 393, in forwardtoken_type_ids = torch.zeros_like(input_ids)
RuntimeError: CUDA error: no kernel image is available for execution on the device

解决方法：change the cuda version
https://blog.csdn.net/li4692625/article/details/123169438

Traceback (most recent call last):File "src/driver/driver.py", line 37, in <module>sim.makeAction(sim, [1], [1], [1])
TypeError: makeAction(): incompatible function arguments. The following argument types are supported:1. (self: MatterSim.Simulator, arg0: List[int], arg1: List[float], arg2: List[float]) -> None

    # For now, the agent can't pick which forward move to make - just the one in the middleenv_actions = {'left': [[0],[-1], [0]], # left'right': [[0], [1], [0]], # right'up': [[0], [0], [1]], # up'down': [[0], [0],[-1]], # down'forward': [[1],[0], [0]], # forward'<end>': [[0], [0], [0]], # <end>'<start>': [[0], [0], [0]], # <start>'<ignore>': [[0], [0], [0]]  # <ignore>}

解决方法：change the defined action ‘left’: (0, 0, 0) to ‘left’: [[0],[-1], [0]].

Result:

Loaded the listener model at iter 114000 from snap/VLNBERT-PREVALENT-final/state_dict/best_val_unseen
result length 1501
Env name: val_train_seen, nav_error: 0.8354, oracle_error: 0.6634, steps: 5.1839, lengths: 10.0262, success_rate: 0.9394, oracle_rate: 0.9520, spl: 0.9125
result length 1021
Env name: val_seen, nav_error: 2.8973, oracle_error: 1.9402, steps: 5.5436, lengths: 11.1393, success_rate: 0.7228, oracle_rate: 0.7826, spl: 0.6774
result length 2349
Env name: val_unseen, nav_error: 3.9281, oracle_error: 2.5456, steps: 6.1235, lengths: 12.0003, success_rate: 0.6275, oracle_rate: 0.7020, spl: 0.5684

Recurrent-VLN-BERT相关推荐

吴琦：AI研究一路走到“黑”，从VQA到VLN
作者 | 吴琦,阿德莱德大学助理教授本文作者为阿德莱德大学助理教授吴琦,他详细介绍有关从VQA到VLN的详细内容: VLN之来龙去脉篇 VLN之任务数据篇 VLN之方法创新篇 VLN之未来思考篇 V ...
VLN阅读报告1：Vision-and-Language Navigation综述（2022ACL）
本博文是结合论文Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions,对VLN进行学习总结 ...
VLN: 基于全局对比训练的视觉-语言导航方法
每天给你送来NLP技术干货! 来自:CAAI认知系统与信息处理专委会视觉-语言导航任务(Vision-Language Navigation, VLN)是指在陌生环境中,无人系统依据语言指示和观测图 ...
CVPR 2021 论文和开源项目合集(Papers with Code)
摘自:https://github.com/amusi/CVPR2021-Papers-with-Code CVPR 2021 论文和开源项目合集 CVPR 2021 论文和开源项目合集(Papers ...
ICLR 2020论文投稿2600篇，GNN、BERT、Transformer领跑热门研究方向
(图片由AI科技大本营付费下载自视觉中国) 出品 | AI科技大本营(ID:rgznai100) 2019 年 4,ICLR 2020 论文征集活动开始,截止 9 月 25 日,大会共收到近 2600 ...
如何使用 BERT 进行自然语言处理？
点击上方"视学算法",选择加"星标"或"置顶" 重磅干货,第一时间送达来源:AI前线本文大约7510字,阅读时间约10分钟.本文介绍并探索 ...
谁说GPT只擅长生成？清华研究力证：GPT语言理解能力不输BERT
点击上方"视学算法",选择加"星标"或"置顶" 重磅干货,第一时间送达作者|张倩.小舟来源|机器之心一直以来,GPT 模型的语言生成能 ...
超越BERT的模型有哪些？
机器之心 & ArXiv Weekly Radiostation 参与:杜伟.楚航.罗若天本周的重要论文包括依图推出的预训练语言理解新模型 ConvBERT,以及关于机器学习模型安全性与隐私 ...
《预训练周刊》第5期：智源等研究力证：GPT语言理解能力不输BERT、盘点NLP预训练「兵器谱」...
No.05 智源社区预训练组预训练研究观点资源活动关于周刊超大规模预训练模型是当前人工智能领域研究的热点,为了帮助研究与工程人员了解这一领域的进展和资讯,智源社区整理了第5期< ...
智源青年科学家杨植麟：为什么预处理模型XLNet比BERT、RoBERTa更加优越
在2020年2月11日下午的"智源论坛Live | 青年科学家线上报告会"中,智源青年科学家.Recurrent AI联合创始人杨植麟做了题为<Latest Advances ...

Recurrent-VLN-BERT

Recurrent VLN-BERT复现遇到的bug

Recurrent-VLN-BERT相关推荐

最新文章

热门文章