Change activation error in WebLogic- Operation TimeOut

Applies for (validated scenarios):

Oracle WebLogic Server- Version 11.1x and above
JDK - Version 1.7x and above
The information in this document applies to any platform

Purpose

Friends, we share a brief advice on synchronization of clustered servers and deployment concepts.

Recently we were called by an exception during the activation of changes (specifically a deployment to a cluster) that generated an exception like the following:

####<Oct 15, 2018 10:55:55,517 AM EDT> <Error> <Console> <apps.rinnovocorp.com> <AdminServer> <[ACTIVE] ExecuteThread: '8' for queue: 'weblogic.kernel.Default (self-tuning)'> <weblogic> <> <b75b1190-2f6a-4ee7-8b6b-5d421e1974ed-00000398> <1539615355517> <[severity-value: 8] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-240003> <Administration Console encountered the following error: java.lang.RuntimeException: Timed out waiting for completion:  Activate State: STATE_DISTRIBUTING Target Servers States: InstanciaB_srv1 STATE_DISTRIBUTED AdminServer STATE_DISTRIBUTED
        at weblogic.management.provider.internal.ActivateTaskImpl.waitForCompletion(ActivateTaskImpl.java:505)
        at weblogic.management.provider.internal.ActivateTaskImpl.waitForTaskCompletion(ActivateTaskImpl.java:478)
        at weblogic.management.provider.internal.EditAccessImpl.activateChangesAndWaitForCompletion(EditAccessImpl.java:1720)
        at weblogic.management.mbeanservers.edit.internal.ConfigurationManagerMBeanImpl.activate(ConfigurationManagerMBeanImpl.java:435)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at weblogic.management.jmx.modelmbean.WLSModelMBean.invoke(WLSModelMBean.java:532)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
        at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
        at weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase$21.run(WLSMBeanServerInterceptorBase.java:589)
        at java.security.AccessController.doPrivileged(Native Method)
        at weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase.invoke(WLSMBeanServerInterceptorBase.java:587)
        at weblogic.management.mbeanservers.internal.JMXContextInterceptor.invoke(JMXContextInterceptor.java:249)
        at weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase$21.run(WLSMBeanServerInterceptorBase.java:589)
        at java.security.AccessController.doPrivileged(Native Method)
        at weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase.invoke(WLSMBeanServerInterceptorBase.java:587)
        at weblogic.management.mbeanservers.edit.internal.RecordingInterceptor.invoke(RecordingInterceptor.java:178)
        at weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase$21.run(WLSMBeanServerInterceptorBase.java:589)
        at java.security.AccessController.doPrivileged(Native Method)
        at weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase.invoke(WLSMBeanServerInterceptorBase.java:587)
        at weblogic.management.mbeanservers.internal.SecurityMBeanMgmtOpsInterceptor.invoke(SecurityMBeanMgmtOpsInterceptor.java:65)
        at weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase$21.run(WLSMBeanServerInterceptorBase.java:589)
        at java.security.AccessController.doPrivileged(Native Method)
        at weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase.invoke(WLSMBeanServerInterceptorBase.java:587)
        at weblogic.management.mbeanservers.edit.internal.EditLockInterceptor.invoke(EditLockInterceptor.java:135)
        at weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase$21.run(WLSMBeanServerInterceptorBase.java:589)
        at java.security.AccessController.doPrivileged(Native Method)
        at weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase.invoke(WLSMBeanServerInterceptorBase.java:587)
        at weblogic.management.mbeanservers.internal.SecurityInterceptor.invoke(SecurityInterceptor.java:439)
        at weblogic.management.jmx.mbeanserver.WLSMBeanServerInterceptorBase$21.run(WLSMBeanServerInterceptorBase.java:589)

Cause

According to the detail of the exception and the characteristics of the environment (2 managed in a cluster / on different physical devices), it is because the date / time was not synchronized between the physical hosts members of the cluster

Solutionn

In this particular case, you must check and adjust date / time of the cluster physical host hosts. You can validate this situation by applying changes in each managed. Similarly, executing an ntp protocol on each server. When setting for example with a command (Linux) date 10150948 [MMddHH21Mi] and restart the environment, you will be able to activate the changes in the cluster members again without any problem.