Issue when creating segments / networks on NSX

Hello everyone,

I have a lab with a vCenter / NSX and i am trying to add Cloudstack ontop. I have followed the docs to configure vCenter with NSX and the vCenter cluster has been added to Cloudstack successfully.

Cloudstack version: 4.21.0.0
NSX Version: 4.2.0.1.0.24210154
vCenter Version: 8.0.3

But when i try to create an isolated network the call fails with error:

unable to create NSX network ad00c442-6a64-4772-ab94-43141cc98337 due to: Failed API call to NSX controller

The strange thing is that in NSX the T1 Gateway is being created

Here are the logs from Cloudstack:

2025-10-27 07:10:55,010 DEBUG [o.a.c.s.NsxApiClient] (DirectAgent-22:[ctx-c6ddeea7]) (logid:ea3ad574) NSX Tier-1 gateway with name: D1-A2-Z1-N224 not found
2025-10-27 07:10:55,195 ERROR [o.a.c.s.NsxApiClient] (DirectAgent-161:[ctx-385a40c3]) (logid:ea3ad574) Error creating segment D1-A2-Z1-S224: General error has occurred.
2025-10-27 07:10:55,195 ERROR [o.a.c.r.NsxResource] (DirectAgent-161:[ctx-385a40c3]) (logid:ea3ad574) Failed to create network: adf
2025-10-27 07:10:55,195 ERROR [o.a.c.u.NsxControllerUtils] (qtp1245065720-24:[ctx-d382e352, ctx-25d4fd50]) (logid:ea3ad574) NSX API Command failed
2025-10-27 07:10:55,219 ERROR [c.c.a.ApiServer] (qtp1245065720-24:[ctx-d382e352, ctx-25d4fd50]) (logid:ea3ad574) unhandled exception executing api command: [Ljava.lang.String;@2edc4927 com.cloud.utils.exception.CloudRuntimeException: unable to create NSX network 2b833a44-68a5-4d63-b449-8129c4074289due to: Failed API call to NSX controller
at org.apache.cloudstack.service.NsxGuestNetworkGuru.setup(NsxGuestNetworkGuru.java:153)
at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.setupNetwork(NetworkOrchestrator.java:838)
at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator$11.doInTransaction(NetworkOrchestrator.java:3098)
at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator$11.doInTransaction(NetworkOrchestrator.java:2995)
at com.cloud.utils.db.Transaction$2.doInTransaction(Transaction.java:48)
at com.cloud.utils.db.Transaction.execute(Transaction.java:38)
at com.cloud.utils.db.Transaction.execute(Transaction.java:45)
at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.createGuestNetwork(NetworkOrchestrator.java:2995)
at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.createGuestNetwork(NetworkOrchestrator.java:2734)
at com.cloud.network.NetworkServiceImpl$4.doInTransaction(NetworkServiceImpl.java:2332)
at com.cloud.network.NetworkServiceImpl$4.doInTransaction(NetworkServiceImpl.java:2270)
at com.cloud.utils.db.Transaction.execute(Transaction.java:38)
at com.cloud.network.NetworkServiceImpl.commitNetwork(NetworkServiceImpl.java:2270)
at com.cloud.network.NetworkServiceImpl.createGuestNetwork(NetworkServiceImpl.java:1830)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:109)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:52)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
at jdk.proxy3/jdk.proxy3.$Proxy200.createGuestNetwork(Unknown Source)
at org.apache.cloudstack.api.command.user.network.CreateNetworkCmd.execute(CreateNetworkCmd.java:431)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:173)
at com.cloud.api.ApiServer.queueCommand(ApiServer.java:854)
at com.cloud.api.ApiServer.handleRequest(ApiServer.java:668)
at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:416)
at com.cloud.api.ApiServlet$1.run(ApiServlet.java:193)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:190)
at com.cloud.api.ApiServlet.doPost(ApiServlet.java:149)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:665)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:750)
at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1450)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:554)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:772)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.base/java.lang.Thread.run(Thread.java:840)

I have also tried to create a VPC which works ok and i can ping the public IP address it allocates but when i try to create a network inside the VPC it fails again with the same error.

So my guess is that there is something going on when Cloudstack tries to create a segment on NSX.

Is this a bug or have i forgotten to configure something?

Reproduire la création d’un réseau isolé sur les versions indiquées de CloudStack, NSX et vCenter, puis suivre l’échec depuis NsxGuestNetworkGuru.java:153 à travers NsxResource et NsxApiClient, en prenant comme point de départ les entrées de journal de création du segment. Comparer la réponse de l’API NSX avec la requête effectuée pour le segment ; le travail est terminé lorsque la cause est identifiée et que la création du réseau réussit ou signale une erreur spécifique et exploitable.

Rédigé par le modèle d'indexation à partir du texte de l'issue.