I'd like to show how to add end-to-end monitoring to your CXF based applications. End-to-end monitoring means that you can follow the message flow which has been triggered by a user across several web services nodes (consumer/provider). Context information of a message and its content are pushed by every web service node to a central server component. The communication style is asynchronous to not delay message processing.
To show how this can be achieved I'll also use the example wsclientWebapp of the Apache CXF Fediz project which I already used to illustrate how to add load balancing and failover. Further information is available here.
This example already supports Web SSO (IDP), WS-Security and STS. The architecture is described in a previous blog.
A web appliation is federation enabled with Fediz to support SSO. As part of the login of the browser user, a SAML token is issued which contains the claims information which are relevant to the web application. The web application calls a web service on behalf of the logged in user. This is accomplished by the CXF STS which is shipped as part of the Fediz in the IDP/STS component. The Web Service Consumer (Web Application) requests a new token based on the WS-SecurityPolicy of the Web Service Provider on behalfof of the security token issued as part of the Web Login. In this example, the security tokens are SAML tokens.
More information on how to build, deploy and test the demo are described in the README.
This example has the following gaps to be deployed in a distributed environment:
- URL's of the web service provider is configured on the service consumer side which is difficult to manage across the environments and for all web service consumers
- If an error occurs you have to analyze log files of all involved nodes in the message flow to drill down the root cause of the issue. Usually, the log files of all involved applications are distributed in the network. It's difficult to correlate the messages in the different log files.
Talend built Apache licensed open source components to add load balancing, failover and end-to-end monitoring to your CXF/Camel based applications. This blog shows you how easily you can integrate this component into your application.
Update your CXF application
Talend built a SAM (Service Activity Monitoring) server component where service participants (consumer/provider) can send information about incoming and outgoing messages. The SAM server is Apache licensed. Besides the server component, the SAM agent is deployed as part of the CXF application which hooks into the interceptor chain to collect all required information about the message and pushes it to the SAM server asynchronously.What do you have to do in your application?
A) Web Service Provider
Make the following changes in the maven project wsclientWebapp/webservice/service
1) add POM dependency for the Talend SAM agent library
This library hooks into the CXF interceptor chain to send service activity information to the SAM server.
<dependency>
<groupId>org.talend.esb</groupId>
<artifactId>sam-agent</artifactId>
<version>5.1.1</version>
</dependency>
2) add a file agent.properties to your maven project src/main/resources
This file contains information where the SAM agent is running and what kind of information should be sent.
collector.scheduler.interval=500 collector.maxEventsPerCall=10 collector.lifecycleEvent=false log.messageContent=true log.maxContentLength=-1 log.enforceMessageIDTransfer=false service.url=http://localhost:8040/services/MonitoringServiceSOAP service.retry.number=3 service.retry.delay=50003) Update Spring configuration file
applicationContext.xml
The highlighted lines must be added to the example application.
...
<import resource="classpath:META-INF/cxf/cxf.xml" />
<import resource="classpath:META-INF/tesb/locator/beans.xml" />
<import resource="classpath:META-INF/tesb/agent-context.xml" />
...
<!-- GreeterService -->
<jaxws:endpoint id="GreeterService"
implementor="org.apache.cxf.fediz.examples.service.GreeterImpl"
wsdlLocation="WEB-INF/wsdl/hello_world.wsdl"
serviceName="svc:GreeterService"
xmlns:svc="http://apache.org/hello_world_soap_http"
address="/GreeterService">
<jaxws:properties>
<entry key="ws-security.signature.properties" value="stsKeystore.properties" />
<entry key="org.talend.tesb.endpoint.secured" value="true"/>
</jaxws:properties>
<!-- Talend feature -->
<jaxws:features>
<bean class="org.talend.esb.servicelocator.cxf.LocatorFeature" />
<ref bean="eventFeature"/>
</jaxws:features>
</jaxws:endpoint>
4) Run "mvn clean install"
Maven builds a new WAR package. This package is now able to send service activity information to the SAM server.
B) Web Service Consumer
Make the following changes in the maven project wsclientWebapp/webapp
1) add POM dependency for the Talend SAM agent library
This library hooks into the CXF interceptor chain to send service activity information to the SAM server.
<dependency>
<groupId>org.talend.esb</groupId>
<artifactId>sam-agent</artifactId>
<version>5.1.1</version>
</dependency>
2) add a file agent.properties to your maven project src/main/resources
This file contains information where the SAM agent is running and what kind of information should be sent.
collector.scheduler.interval=500 collector.maxEventsPerCall=10 collector.lifecycleEvent=false log.messageContent=true log.maxContentLength=-1 log.enforceMessageIDTransfer=false service.url=http://localhost:8040/services/MonitoringServiceSOAP service.retry.number=3 service.retry.delay=5000There are different options to configure what kind of information should be pushed to the SAM server. Please check the Talend_ESB_InfrastructureServices manual for all configuration options like:
- Send message content
- Configure maximum length of content to be sent
- filters to find/replace in message content
- add custom context information
- ...
applicationContext.xml
The highlighted lines must be added to the example application.
...
<import resource="classpath:META-INF/cxf/cxf.xml" />
<import resource="classpath:META-INF/tesb/locator/beans.xml" />
<import resource="classpath:META-INF/tesb/agent-context.xml" />
...
<jaxws:client id="HelloServiceClient" serviceName="svc:GreeterService"
xmlns:svc="http://apache.org/hello_world_soap_http"
serviceClass="org.apache.hello_world_soap_http.Greeter"
address="locator://whatever"
wsdlLocation="WEB-INF/wsdl/hello_world.wsdl">
<jaxws:properties>
<entry key="ws-security.sts.client">
<bean class="org.apache.cxf.ws.security.trust.STSClient">
<constructor-arg ref="cxf" />
<property name="wsdlLocation" value="https://localhost:9443/fedizidpsts/STSServiceTransport?wsdl" />
<property name="serviceName"
value="{http://docs.oasis-open.org/ws-sx/ws-trust/200512/}SecurityTokenService" />
<property name="endpointName"
value="{http://docs.oasis-open.org/ws-sx/ws-trust/200512/}Transport_Port" />
<property name="onBehalfOf" ref="delegationCallbackHandler" />
<property name="enableAppliesTo" value="true" />
</bean>
</entry>
<entry key="ws-security.cache.issued.token.in.endpoint" value="false" />
</jaxws:properties>
<jaxws:features>
<bean class="org.talend.esb.servicelocator.cxf.LocatorFeature" />
<ref bean="eventFeature"/>
</jaxws:features>
</jaxws:client>
4) Run "mvn clean install"
Maven builds a new WAR package. This package is now able to send service activity information to the SAM server.
Deploy SAM server
Last but not least, the Talend SAM server must be deployed and started which is shipped as part of the Talend ESB. Follow these steps to start the SAM server (if you have run already the demo for the Service Locator you only have to runtesb:start-all):
- Download the Talend ESB Standard Edition (SE) here. The Standard Edition has full functionality and is Apache licensed.
- Unzip the file
- Run
<install-dir>/container/trun - Execute the following command in the console
tesb:start-all - You can log the most recent logs with the command
log:display
The SAM server (and Service Locator) are running now.
More information about the Talend ESB is available here:
Test the application
You can test the application as described in the README of the Fediz example wsclientWebapp.Enter the URL https://localhost:8443/fedizhelloworld/secure/service.jsp and click on the button Call Service after login (User: alice, Password: ecila). This triggers a web service call.
You will also see that messages are sent to the SAM server from the service consumer and service provider:
... Jul 5, 2012 11:21:09 PM org.talend.esb.sam.agent.collector.EventCollector sendEvents INFO: Put events(2) to Monitoring Server. ...
You can use any DB Visualizer Tool to show the data of the messages exchanged in the web services network. In this example, the data is written to Apache Derby.
You find more information where this DB is located in Talend_ESB_InfrastructureServices in chapter 4.3.
This is not a very nice UI to see which messages are part of a flow triggered by a user. In my next blog I'll describe the Talend Administration Console (TAC) which allows to see the messages exchanged and the registered service in a graphical way.
This blog showed how easy it is to add end-to-end monitoring capabilities to your CXF application with open source components and without changing a single line of code.
