Updated Megatron version#85

Draft

jlamypoirier wants to merge 437 commits intobigcode-project:nvidia_mainfrom

ServiceNow:compare_tensors_updated

This pull request is big! We're only showing the most recent 250 commits

Commits on Jan 18, 2024

Commits on Jan 19, 2024

Commits on Jan 20, 2024

Commits on Jan 21, 2024

Code clean.
yanring
committed

Commits on Jan 22, 2024

Commits on Jan 23, 2024

Commits on Jan 24, 2024

Commits on Jan 25, 2024

Update s_app_tag with {job_name}_{batch_size}_{gpu_req}
PytLab
committed
Merge branch 'offload_patch' into 'main'
jaredcasper
committed
Log metrics in consistent order
PytLab
committed
Add app_tag_count tracking
PytLab
committed
Merge branch 'feature/add-e2e-metrics-logging' of ssh://gitlab-master.nvidia.com:12051/zshao/megatron-lm into feature/add-e2e-metrics-logging
Zhengjiang Shao
committed
Resolve merging conflict
Zhengjiang Shao
committed
Use app tag logging wrapper api
PytLab
committed
Remove app_tag global var
PytLab
committed
Merge branch 'main' into feature/add-e2e-metrics-logging
PytLab
committed
Add doc
mikolajblaz
committed
Add no support info
mikolajblaz
committed
Adding bert local spec test
Shanmugam Ramasamy
committed
Adding bert local spec test
Shanmugam Ramasamy
committed
Merge branch 'zijiey/moe_api_clean' into 'main'
jaredcasper
committed
Adding bert local spec test
Shanmugam Ramasamy
committed

Commits on Jan 26, 2024

Commits on Jan 29, 2024

Merge branch 'main' into mblaz/moe-0.5-dist-ckpt
mikolajblaz
committed
Adding bert local spec test
Shanmugam Ramasamy
committed

Commits on Jan 30, 2024

Commits on Feb 1, 2024

Commits on Feb 3, 2024

Commits on Feb 6, 2024

MoE argument sanity checks

akoumpa
authored and
ericharper
committed
Merge branch 'akoumparouli/arg_sanity_check' into 'main'
ericharper
committed
add add_qkv_bias config

xueh-nv
authored and
jaredcasper
committed
Merge branch 'xueh/add_qkv_bias' into 'main'
jaredcasper
committed
Minor fixes for JET CI

maanug-nv
authored and
jaredcasper
committed
Merge branch 'maanug/jet-minor-fixes' into 'main'
jaredcasper
committed
Tokenizer fix
jlamypoirier
committed
Merge remote-tracking branch 'nvidia/main' into compare_tensors_updated
jlamypoirier
committed
Check if config has num_moe_experts

akoumpa
authored and
jaredcasper
committed
Merge branch 'akoumparouli/moe_config_check' into 'main'
jaredcasper
committed
Add dist ckpt package docs for Sphinx documentation

mikolajblaz
authored and
jaredcasper
committed
Merge branch 'mblaz/dist-ckpt-docs' into 'main'
jaredcasper
committed
Fix oob perf

wdykas
authored and
jaredcasper
committed
Merge branch 'fix-oob-perf' into 'main'
jaredcasper
committed
Add interleaved rotary embedding in MCore

xueh-nv
authored and
jaredcasper
committed
Merge branch 'xueh/rotary_interleaved' into 'main'
jaredcasper
committed
fix activation checkpointing mutation

gshennvm
authored and
ericharper
committed
Merge branch 'geshen/fix_activation_mutation' into 'main'
ericharper
committed
fix
jlamypoirier
committed

Commits on Feb 7, 2024

Commits on Feb 9, 2024

Adding back the changes needed in timers.py for E2E work
Ankur Joshi
committed

Commits on Feb 10, 2024

Fixed atomic gemm defaults/fixed the offloading check
Selvaraj Anandaraj
committed

Commits on Feb 11, 2024

Put embedding layers in separate buckets to make sure embedding tying works
deepakn94
committed

Commits on Feb 12, 2024

Ran black(19.10b0) on megatron/core
Ankur Joshi
committed
Use MCore for distributed optimizer tests
deepakn94
committed

Commits on Feb 14, 2024

Commits on Feb 15, 2024

Fixing examples
Shanmugam Ramasamy
committed
Merge branch 'bugfixexample' into 'main'
jaredcasper
committed

Commits on Feb 21, 2024

[MoE] Expert data parallel w/ ZeRO-1 support

shjwudp
authored and
ericharper
committed
Merge branch 'edp_with_zero1' into 'main'
ericharper
committed

Commits on Feb 22, 2024

Merge branch 'config_default' into 'main'
jaredcasper
committed

Commits on Feb 23, 2024

Make sure data_end_index is padded when creating new buckets
deepakn94
committed

Commits on Feb 24, 2024

Commits on Feb 26, 2024

Print number of transformer and embedding parameters separately
deepakn94
committed

Commits on Feb 28, 2024

Commits on Feb 29, 2024

Commits on Mar 3, 2024

Merge branch 'variable_ffn_size' into 'main'
deepakn94
committed

Commits on Mar 5, 2024

Experimental Yaml configs

wdykas
authored and
jaredcasper
committed
Merge branch 'yaml' into 'main'
jaredcasper
committed

Commits on Mar 8, 2024

Commits on Mar 11, 2024

Commits on Mar 12, 2024

fixes
jlamypoirier
committed

Commits on May 29, 2024

fix
jlamypoirier
committed

Commits on Sep 19, 2024

fix
jlamypoirier
committed

Commits on Feb 11, 2025

update
jlamypoirier
committed

Commits on Mar 5, 2025

misc
jlamypoirier
committed

Commits on May 16, 2025

Commits on Jul 23, 2025

fix
jlamypoirier
committed

Commits on Aug 14, 2025

misc
jlamypoirier
committed

Commits on Sep 3, 2025

stuff
jlamypoirier
committed

Commits on Sep 9, 2025

fix
jlamypoirier
committed

Commits on Oct 2, 2025

misc
jlamypoirier
committed