Issues

• 执行hf转megatron格式报错了

  alibaba/Megatron-LLaMA#70 · Filter by author Lilypad97 opened on Oct 21, 2024

• TypeError: perform_nocolor_split(): incompatible function arguments.

  alibaba/Megatron-LLaMA#69 · Filter by author vedantgoswami opened on Aug 14, 2024
  1

• When training BERT, ERROR: "AttributeError: 'FullTokenizer' object has no attribute 'save_pretrained' "

  alibaba/Megatron-LLaMA#68 · Filter by author yuzhiguo07 opened on Aug 14, 2024

• 如何断点续训

  alibaba/Megatron-LLaMA#66 · Filter by author MAxx8371 opened on Jul 4, 2024

• No update for a long time

  alibaba/Megatron-LLaMA#65 · Filter by author dong-liuliu opened on Jun 21, 2024

• Has llama2 GQA been supported yet?

  alibaba/Megatron-LLaMA#64 · Filter by author JiwenJ opened on May 26, 2024

• 请问有dingding群聊，或者微信群吗？可以沟通的

  alibaba/Megatron-LLaMA#63 · Filter by author felix0080 opened on May 24, 2024

• Llama 3 Support

  alibaba/Megatron-LLaMA#62 · Filter by author john-theo opened on May 3, 2024

• About batch_size

  alibaba/Megatron-LLaMA#61 · Filter by author tszslovewanpu opened on Mar 16, 2024

• sh LLaMA2_7B_standalone.sh

  alibaba/Megatron-LLaMA#60 · Filter by author yangzhipeng1108 opened on Mar 14, 2024

• 请问是否支持从0训练一个小规模的LLaMA模型，如：1B

  alibaba/Megatron-LLaMA#59 · Filter by author liubo12 opened on Mar 1, 2024
  1

• 注意力权重转换问题

  alibaba/Megatron-LLaMA#58 · Filter by author noob-ctrl opened on Feb 19, 2024
  2

• 在模型转换权重时遇到了如下问题 Zarr-based strategies will not be registered because of missing packages

  alibaba/Megatron-LLaMA#57 · Filter by author ZhangEnmao opened on Feb 5, 2024

• 使用distributed optimzer时grad_norm计算准确度的疑问

  alibaba/Megatron-LLaMA#56 · Filter by author chivychao opened on Dec 27, 2023
  1

• LLaMAModel._causal_lm_process中的labels和logits对齐方法疑问

  alibaba/Megatron-LLaMA#53 · Filter by author chivychao opened on Dec 8, 2023
  3

• Megatron-LM权重转hf格式

  alibaba/Megatron-LLaMA#52 · Filter by author Yang-QW opened on Dec 2, 2023
  4

• 问下readme中32机的吞吐对应的参数可以提供下吗，目前没有复现出来

  alibaba/Megatron-LLaMA#49 · Filter by author jianzi123 opened on Nov 8, 2023
  5

• 求一份Serving的教程代码

  alibaba/Megatron-LLaMA#48 · Filter by author xealml opened on Nov 8, 2023
  1

• hf权重转换代码小bug

  alibaba/Megatron-LLaMA#47 · Filter by author yuanzhoulvpi2017 opened on Nov 3, 2023
  

• INT4 量化的模型可以被Megatron-LLaMA支持吗？

  alibaba/Megatron-LLaMA#46 · Filter by author Jeff123z opened on Oct 17, 2023
  1

• 请问目前Megatron-LLaMA支持LLaMA2-70B的训练吗？

  alibaba/Megatron-LLaMA#45 · Filter by author 13416157913 opened on Oct 17, 2023
  1

• fp16的支持问题

  alibaba/Megatron-LLaMA#41 · Filter by author XUWeijiang opened on Oct 9, 2023
  1

• OverlappedDistributedOptimizer 支持 pipeline parallelism > 1 和 data parallelism > 1 同时使用吗？

  alibaba/Megatron-LLaMA#37 · Filter by author Baibaifan opened on Sep 26, 2023
  8

• 对于不同参数模型，如何通过配置参数信息计算显存占用大小？

  alibaba/Megatron-LLaMA#36 · Filter by author 13416157913 opened on Sep 26, 2023
  1

• 大家好，请教一个关于GLOBAL_BATCH_SIZE值计算的问题，希望大家不吝赐教。

  alibaba/Megatron-LLaMA#35 · Filter by author 13416157913 opened on Sep 25, 2023
  7