Performance Issue with NVIDIA Transformer Engine FP8 Linear Functions on L20

issue

When testing the linear API provided by NVIDIA's transformer engine (with FP8 precision) on an L20 device, I found that its speed is significantly slower than PyTorch's built-in linear API. Is there something I might have missed in the configuration or additional optimization settings required? Any suggestions or guidance would be appreciated.

code

`import torch
import transformer_engine.pytorch as te
from transformer_engine.pytorch import fp8_model_init
from transformer_engine.common import recipe
import time

in_features = 768
out_features = 3072
hidden_size = 2048
iters = 5

with fp8_model_init(enabled=True):
model = te.Linear(in_features, out_features, bias=True, device="cuda")
inp = torch.randn(hidden_size, in_features, device="cuda")
for name, param in model.named_parameters():
print(f"{name}: {param.dtype}")
torch_model = torch.nn.Linear(in_features, out_features, bias=True).cuda()
for name, param in torch_model.named_parameters():
print(f"{name}: {param.dtype}")

fp8_recipe = recipe.DelayedScaling(margin=0, fp8_format=recipe.Format.E4M3)

for _ in range(5):
out = torch_model(inp)

times = []
with torch.no_grad():
for i in range(iters):
start = time.time()
with te.fp8_autocast(enabled=True, fp8_recipe=fp8_recipe):
out = model(inp) # input改成inp
torch.cuda.synchronize()
end = time.time()
times.append(end - start)
avg_time_te = sum(times) / len(times)
print(f"te linear average costime: {avg_time_te:.6f} seconds")

torch_times = []
for i in range(iters):
start = time.time()
with torch.no_grad():
out = torch_model(inp)
torch.cuda.synchronize()
end = time.time()
torch_times.append(end - start)
avg_time_torch = sum(torch_times) / len(torch_times)
print(f"torch linear average costime: {avg_time_torch:.6f} seconds")`

result

environment

Start by reproducing the supplied Python benchmark comparing transformer_engine.pytorch te.Linear with torch.nn.Linear on the NVIDIA L20 setup described in the environment image. Inspect the FP8 model initialization, DelayedScaling recipe, autocast context, warmup, and timing loop; done means identifying whether configuration or an implementation issue explains the performance gap and documenting actionable guidance.