Hi Support team,
we have created a new system profile and are now trying to install 2 web server agents.
The agents are also displayed in the agents overview, but not be mapped to the system profile. We have checked the names twice.
Here are the logging information:
2014-10-15 15:54:52.878662 [16752/a8c638bb] info [native] Starting dynaTrace Web Server Agent (pid = 16752)
2014-10-15 15:54:52.878705 [16752/a8c638bb] info [native] Reading configuration file: /DBA/nest/apache/WSAIREA2/dynatrace/dynatrace-6.0.0/agent/conf/dtwsagent.ini
2014-10-15 15:54:52.879038 [16752/a8c638bb] info [native] Initialized logfile: /DBA/nest/apache/WSAIREA2/dynatrace/dynatrace-6.0.0/log/dt_dtwsagent_16752.log
2014-10-15 15:54:52.879089 [16752/a8c638bb] info [native] Max. logfile size: 10485760 bytes
2014-10-15 15:54:52.879098 [16752/a8c638bb] info [native] No sharedmemoryfilename directive supplied in configuration. Using default.
2014-10-15 15:54:52.879105 [16752/a8c638bb] info [native] No port directive supplied in configuration. Using default: 0
2014-10-15 15:54:52.879298 [16752/a8c638bb] info [native] Using SHM location: /DBA/nest/apache/WSAIREA2/dynatrace/dynatrace-6.0.0/agent/conf/dynaTraceWebServerSharedMemory
2014-10-15 15:54:52.879331 [16752/a8c638bb] warning [native] SharedMemory::attachOrCreate: Could not change the file access permissions of /DBA/nest/apache/WSAIREA2/dynatrace/dynatrace-6.0.0/agent/conf/dynaTraceWebServerSharedMemory: Operation not permitted
2014-10-15 15:54:52.879347 [16752/a8c638bb] info [native] Reinitializing config data in shared memory segment of file /DBA/nest/apache/WSAIREA2/dynatrace/dynatrace-6.0.0/agent/conf/dynaTraceWebServerSharedMemory.
2014-10-15 15:54:52.879435 [16752/a8c638bb] info [native] Starting agent real_Webserver_I2
2014-10-15 15:54:52.879630 [16752/a8c638bb] info [native] Loading collector peer list from /DBA/nest/apache/WSAIREA2/dynatrace/dynatrace-6.0.0/agent/conf/collectorlist.real_Webserver_I2
2014-10-15 15:54:53.180167 [16752/a8c638bb] info [native] Using proc cpu time provider
2014-10-15 15:54:53.180325 [16752/a8c638bb] info [native] -----------------------------------------------------------------------------------------------
2014-10-15 15:54:53.180336 [16752/a8c638bb] info [native] dynaTrace Agent for Web server Copyright (c) 2004-2014 Compuware Corporation, www.compuware.com
2014-10-15 15:54:53.180342 [16752/a8c638bb] info [native] -----------------------------------------------------------------------------------------------
2014-10-15 15:54:53.180350 [16752/a8c638bb] info [native] Version ..................... 6.0.0.7000, build date Sep 1 2014 13:35:54
2014-10-15 15:54:53.180374 [16752/a8c638bb] info [native] Command line is /DBA/nest/apache/WSAIREA2/dynatrace/dynatrace-6.0.0/agent/lib64/dtwsagent server=xxx.xxx.xxx.xxx
2014-10-15 15:54:53.180381 [16752/a8c638bb] info [native] Build platform .............. Linux x86 64-bit
2014-10-15 15:54:53.180389 [16752/a8c638bb] info [native] Platform .................... Linux 3.0.101-0.35-default x86_64
2014-10-15 15:54:53.180395 [16752/a8c638bb] info [native] Bootstrap version ........... 6.0.0.6738
2014-10-15 15:54:53.180401 [16752/a8c638bb] info [native] Log level console ........... none
2014-10-15 15:54:53.180406 [16752/a8c638bb] info [native] Log level file .............. info
2014-10-15 15:54:53.180412 [16752/a8c638bb] info [native] Log file .................... /DBA/nest/apache/WSAIREA2/dynatrace/dynatrace-6.0.0/log/dt_dtwsagent_16752.0.log
2014-10-15 15:54:53.180420 [16752/a8c638bb] info [native] Agent host .................. degtluv4048
2014-10-15 15:54:53.180441 [16752/a8c638bb] info [native] Agent library ............... /DBA/nest/apache/WSAIREA2/dynatrace/dynatrace-6.0.0/agent/downloads/6.0.0.7000/native/linux-x86-64/libdtwsagent.so
2014-10-15 15:54:53.180448 [16752/a8c638bb] info [native] Agent name .................. real_Webserver_I2
2014-10-15 15:54:53.180455 [16752/a8c638bb] info [native] Server/Collector ............ xxx.xxx.xxx.xxx:9998
2014-10-15 15:54:53.183482 [16752/a8c638bb] info [native] Reading LoadBalancing configuration from xxx.xxx.xxx.xxx:9998
2014-10-15 15:54:53.183500 [16752/a8c638bb] info [native] Storing collector peer list (0 entries) to /DBA/nest/apache/WSAIREA2/dynatrace/dynatrace-6.0.0/agent/conf/collectorlist.real_Webserver_I2
2014-10-15 15:54:53.183599 [16752/a8c638bb] warning [native] Exception storing collector peer list: Permission denied
2014-10-15 15:54:53.183639 [16752/a8c638bb] info [native] Trying to connect to Server/Collector for up to 19 seconds
2014-10-15 15:54:53.193346 [16752/a8c638bb] warning [native] Starting without instrumentation, agent did not match a System Profile.
2014-10-15 15:54:53.193613 [16752/a8c638bb] info [native] Instrumentation channel connected successfully
2014-10-15 15:54:53.193629 [16752/a8c638bb] info [native] Connected to Server/Collector xxx.xxx.xxx.xxx:9998
2014-10-15 15:54:53.198694 [16752/a71cf8bb] info [native] Control channel connected successfully
2014-10-15 15:54:53.206885 [16752/a71cf8bb] info [native] Agent ID .................... 22885ceb
2014-10-15 15:54:53.206903 [16752/a71cf8bb] info [native] Process ID .................. 16752
2014-10-15 15:54:53.206909 [16752/a71cf8bb] info [native] Capture ..................... disabled
2014-10-15 15:54:53.206915 [16752/a71cf8bb] info [native] Capture CPU times ........... disabled
2014-10-15 15:54:53.208389 [16752/a71cf8bb] info [native] Hypervisor .................. VMware
2014-10-15 15:54:53.309600 [16752/a8c638bb] info [native] Eventsender channel connected successfully
2014-10-15 15:54:53.330313 [16752/a8c638bb] info [native] Successfully started agent real_Webserver_I2
2014-10-15 15:55:03.069178 [16752/a3e568bb] info [native] New subAgent registered successfully: 6cd72d50
2014-10-15 15:55:03.069226 [16752/a3e568bb] info [native] sub agent registered with id (sub/slave) 6cd72d50/76f5e9733
2014-10-15 16:43:06.426053 [16752/a71cf8bb] info [native] Sending log files (including previous)
2014-10-15 16:52:12.332583 [16752/a71cf8bb] severe [native] Exception in controller: util::SocketConnection::receiveExact(), 70014, End of file found. Retrying every 10 seconds.
2014-10-15 16:52:13.333730 [16752/a79ce8bb] warning [native] Instrumentation channel disconnected: server did not reply to ping request
2014-10-15 16:52:13.441642 [16752/a56538bb] warning [native] Exception occurred while sending events: sendExactOnce(), 32, Broken pipe
2014-10-15 16:52:25.442562 [16752/a79ce8bb] info [native] Instrumentation channel connected successfully
2014-10-15 16:52:25.442683 [16752/a79ce8bb] info [native] Connected to Server/Collector xxx.xxx.xxx.xxx:9998
2014-10-15 16:52:25.442697 [16752/a79ce8bb] info [native] Instrumentation channel: Sending 11 Transformations
2014-10-15 16:52:25.470561 [16752/a56538bb] info [native] Eventsender channel connected successfully
2014-10-15 16:52:25.470586 [16752/a56538bb] info [native] Event channel connected successfully
2014-10-15 16:52:25.471206 [16752/a79ce8bb] info [native] Instrumentation channel: reregistering 2 subAgents
2014-10-15 16:52:25.473561 [16752/a79ce8bb] info [native] SubAgent 6cd72d50 reregistered successfully
2014-10-15 16:52:32.381672 [16752/a71cf8bb] info [native] Control channel connected successfully
2014-10-15 16:52:32.399909 [16752/a71cf8bb] info [native] Agent ID .................... 22885ceb
2014-10-15 16:52:32.399925 [16752/a71cf8bb] info [native] Process ID .................. 16752
2014-10-15 16:52:32.399932 [16752/a71cf8bb] info [native] Capture ..................... disabled
2014-10-15 16:52:32.399938 [16752/a71cf8bb] info [native] Capture CPU times ........... disabled
2014-10-16 06:15:04.028660 [16752/a3e568bb] info [native] New subAgent registered successfully: f902444b
2014-10-16 06:15:04.028683 [16752/a3e568bb] info [native] sub agent registered with id (sub/slave) f902444b/7fe23061e
2014-10-16 06:15:29.069468 [16752/a8c638bb] info [native] Unregistering subAgent 6cd72d50
2014-10-16 06:15:29.069542 [16752/a8c638bb] info [native] sub agent unregistered with id (sub/slave) 6cd72d50/76f5e9733
2014-10-16 10:33:54.680904 [16752/a71cf8bb] info [native] Sending log files (including previous)
Any idea what we did wrong?
Thanks and best regards,
Roman
Answer by Roman Z. ·
We have found the error! A reboot has helped to identify it.
The developers had changed the specified agent names a bit during installation. And we thought that it is enough to adapt the system profile. It is not.
After the restart, everything runs perfectly.
Sorry for the confusion.
BR, Roman
Answer by Andreas G. ·
In your System Profile - do you have an Agent Group/Tier configured that has a valid mapping for APCIHYBREA?
To validate that - open your System Profile -> expand the Tier and then click on Agent Mapping. There you will find a list of Agent Names that dynaTrace expects. If there is no mapping for APCIHYBREA1 then you will get the warning "agent did not match system profile"
To fix this you can either add a new mapping or change the agent name on the command line string to use whatever name you currently have configured in your system profile
Feel free to attach a screenshot of your sys profile - or - even export it and attach it
Answer by Dennis M. ·
Hi,
anyone knows what and why the message
"Exception storing collector peer list: Permission denied"
(seen also in the above log) is occuring?
Is this something to worry about and should be fixed? Appears to be a right level thing? Maybe the user running the collector does not have particular rights to write in that peer list?
hi Dennis,
this exception is about the collector-balancing feature where collectors can be arranged in groups for basic failover capabilities. every agent then gets a list of all the collectors in the group which he will persist on-disk.
in this case the user under which this agent/process was started does not have permissions to write into the <install_dir>/agent/conf directory to persist the collector list. usually the agent installer takes care of making those directories read/writeable for everyone, so I guess that maybe the permissions were changed afterwards.
HTH,
Christian
Answer by Roman Z. ·
Thanks Herwig,
the tomcat agent, we have just installed is also not mapped:
2014-10-16 14:09:52 [15a6889a] info [native] Loading collector peer list from /DBA/nest/apc/APCIHYBREA1/dynatrace-6.0.0/agent/conf/collectorlist.APCIHYBREA1
2014-10-16 14:09:52 [15a6889a] info [native] CPU0: cpu MHz: 2599.9990 (min/max MHz: 0.0000/ 0.0000) cpu has TSC: yes, ConstantTSC: yes
2014-10-16 14:09:52 [15a6889a] info [native] CPU1: cpu MHz: 2599.9990 (min/max MHz: 0.0000/ 0.0000) cpu has TSC: yes, ConstantTSC: yes
2014-10-16 14:09:52 [15a6889a] info [native] High-resolution hardware timer detected.
2014-10-16 14:09:52 [15a6889a] info [native] -----------------------------------------------------------------------------------------
2014-10-16 14:09:52 [15a6889a] info [native] dynaTrace Agent for Java Copyright (c) 2004-2014 Compuware Corporation, www.compuware.com
2014-10-16 14:09:52 [15a6889a] info [native] -----------------------------------------------------------------------------------------
2014-10-16 14:09:52 [15a6889a] info [native] Version ..................... 6.0.0.7000, build date Sep 1 2014 13:35:54
2014-10-16 14:09:52 [15a6889a] info [native] Command line is /usr/java/jdk1.6.0_45/jre/bin/java -Xmx3G -Xms3G -XX:PermSize=300M -XX:MaxPermSize=300M -XX:+UseAdaptiveGCBoundary -XX:+UseParallelOldGC -XX:+UseParallelGC -XX:+ExplicitGCInvokesConcurrent -XX:+PrintGCTimeStamps -XX:+PrintGCDetails -Xloggc:/REAL/TEST/hybris/log/tomcat/java_gc.log -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dorg.tanukisoftware.wrapper.WrapperManager.mbean=true -Duser.language=en -Duser.country=US -Duser.region= -Duser.timezone=CET -Djava.endorsed.dirs=../lib/endorsed -Dcatalina.base=.. -Dcatalina.home=.. -Dfile.encoding=UTF-8 -Dlog4j.configuration=log4j_init_tomcat.properties -Djava.util.logging.config.file=jdk_logging.properties -Djava.io.tmpdir=/REAL/TEST/hybris/temp/hybris -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Duseconfig= -DHYBRIS_BIN_DIR=/REAL/TEST/hybris/bin -DHYBRIS_CONFIG_DIR=/REAL/TEST/hybris/config -DHYBRIS_DATA_DIR=/REAL/TEST/hybris/data -DHYBRIS_LOG_DIR=/REAL/TEST/hybris/log -DHYBRIS_TEMP_DIR=/REAL/TEST/hybris/temp/hybris -Djavax.net.ssl.trustStore=/REAL/TEST/hybris/config/cacerts -Djavax.net.ssl.trustStorePassword=changeit -agentpath:/DBA/nest/apc/APCIHYBREA1/dynatrace-6.0.0/agent/lib64/libdtagent.so=name=APCIHYBREA1,server=xxx.xxx.xxx.xxx:9998 -Ddeployed.server.type=tomcat -Djava.library.path=../lib -classpath ../lib/wrapper-3.5.13.jar:../bin/bootstrap.jar -Dwrapper.key=u3ZN6KV80Si7ijqa6PovroEa-XIU-fXe -Dwrapper.port=32002 -Dwrapper.jvm.port.min=31000 -Dwrapper.jvm.port.max=31999 -Dwrapper.disable_console_input=TRUE -Dwrapper.pid=6887 -Dwrapper.version=3.5.13-st -Dwrapper.native_library=wrapper -Dwrapper.service=TRUE -Dwrapper.cpu.timeout=30 -Dwrapper.jvmid=2 -Dwrapper.lang.domain=wrapper org.tanukisoftware.wrapper.WrapperSimpleApp org.apache.catalina.startup.Bootstrap start
2014-10-16 14:09:52 [15a6889a] info [native] Build platform .............. Linux x86 64-bit
2014-10-16 14:09:52 [15a6889a] info [native] Platform .................... Linux 3.0.101-0.35-default x86_64
2014-10-16 14:09:52 [15a6889a] info [native] Bootstrap version ........... 6.0.0.6738
2014-10-16 14:09:52 [15a6889a] info [native] Log level console ........... info
2014-10-16 14:09:52 [15a6889a] info [native] Log level file .............. info
2014-10-16 14:09:52 [15a6889a] info [native] Log file .................... /DBA/nest/apc/APCIHYBREA1/dynatrace-6.0.0/log/dt_APCIHYBREA1_7248.0.log
2014-10-16 14:09:52 [15a6889a] info [native] Agent host .................. degtluv4049
2014-10-16 14:09:52 [15a6889a] info [native] Agent library ............... /DBA/nest/apc/APCIHYBREA1/dynatrace-6.0.0/agent/downloads/6.0.0.7000/native/linux-x86-64/libdtagentcore.so
2014-10-16 14:09:52 [15a6889a] info [native] Options string .............. name=APCIHYBREA1,server=xxx.xxx.xxx.xxx:9998
2014-10-16 14:09:52 [15a6889a] info [native] Agent name .................. APCIHYBREA1
2014-10-16 14:09:52 [15a6889a] info [native] Server/Collector ............ xxx.xxx.xxx.xxx:9998
2014-10-16 14:09:52 [15a6889a] info [native] Interface type .............. JVMTI
2014-10-16 14:09:52 [15a6889a] info [native] Process ID .................. 7248
2014-10-16 14:09:52 [15a6889a] info [native] JVM name .................... Java HotSpot(TM) 64-Bit Server VM 20.45-b01
2014-10-16 14:09:52 [15a6889a] info [native] JVM vendor .................. Sun Microsystems Inc.
2014-10-16 14:09:52 [15a6889a] info [native] JVM info .................... mixed mode, sharing
2014-10-16 14:09:52 [15a6889a] info [native] Detected application server: Apache Tomcat
2014-10-16 14:09:52 [15a6889a] info [native] Reading LoadBalancing configuration from xxx.xxx.xxx.xxx:9998
2014-10-16 14:09:52 [15a6889a] info [native] Storing collector peer list (0 entries) to /DBA/nest/apc/APCIHYBREA1/dynatrace-6.0.0/agent/conf/collectorlist.APCIHYBREA1
2014-10-16 14:09:52 [15a6889a] info [native] Trying to connect to Server/Collector for up to 19 seconds
2014-10-16 14:09:52 [15a6889a] warning [native] Starting without instrumentation, agent did not match a System Profile.
2014-10-16 14:09:52 [15a6889a] info [native] Instrumentation channel connected successfully
2014-10-16 14:09:52 [15a6889a] info [native] Received instrumentation blacklist (0 classes, 0 KiB)
2014-10-16 14:09:52 [15a6889a] info [native] Connected to Server/Collector xxx.xxx.xxx.xxx:9998
2014-10-16 14:09:53 [15a6889a] info [native] Agent classes defined successfully
2014-10-16 14:09:53 [15a6889a] info [native] Platform .................... Linux 3.0.101-0.35-default, amd64
2014-10-16 14:09:53 [15a6889a] info [java ] Java Agent Version .......... 6.0.0.7000, Agent15
2014-10-16 14:09:53 [0606b89a] info [native] Control channel connected successfully
2014-10-16 14:09:53 [0606b89a] info [native] Agent ID .................... 565461ce
2014-10-16 14:09:53 [0606b89a] info [native] Process ID .................. 7248
2014-10-16 14:09:53 [0606b89a] info [native] Capture ..................... disabled
2014-10-16 14:09:53 [0606b89a] info [native] Capture CPU times ........... disabled
2014-10-16 14:09:53 [0606b89a] info [native] Hot Sensor Placement ........ not available
2014-10-16 14:09:53 [0606b89a] info [native] Hypervisor .................. VMware
2014-10-16 14:09:53 [15a6889a] info [native] Eventsender channel connected successfully
2014-10-16 14:09:57 [0606b89a] info [native] License = skipped by license check - agent did not match system profile;
2014-10-16 14:41:37 [0606b89a] info [native] Sending log files (including previous)
2014-10-16 14:44:53 [0686a89a] info [native] Sampling Cache (0 methods) cleared ...
2014-10-16 14:50:15 [0606b89a] info [native] Sending log files (including previous)
Could it be a license problem?
Best regards, Roman
Answer by Herwig R. ·
Roman,
I would open a ticket for that. One thing is confusing me is the Exception thrown in the controller. Maybe a loss of network connection.
Exception in controller: util::SocketConnection::receiveExact(), 70014, End of file found. Retrying every 10 seconds.
Regards, Herwig
JANUARY 15, 3:00 PM GMT / 10:00 AM ET