SAP APIM external Logging using New Relic

I have used rest api call for sending the logs to new relic similar to https://community.sap.com/t5/technology-blog-posts-by-members/splunk-part-1-sap-apim-logging-monitoring/ba-p/13444151

You can check here for payload format and limitations https://docs.newrelic.com/docs/logs/log-api/introduction-log-api/

Newrelic trial account setup
You can create new relic account here.https://newrelic.com/

Create an application key

Copy your and check the logging functionality by testing from postman. Url is different for US and EU. I am using EU url.

https://docs.newrelic.com/docs/logs/log-api/introduction-log-api/

Curl

curl –location ‘https://log-api.eu.newrelic.com/log/v1’
–header ‘Api-Key: YourKey’
–header ‘Content-Type: application/json’
–data ‘{
“message”: “Postman Testing”,
“Proxyname”: “SAP SCN Blogs”,
“level”: “Error”
}’

Message in Relic logs. Filter the mesasges logs based on the timeline. New Relic account setup is done and next is to send the logs from api proxy.

API Proxy Configuration

For demonstration purposes, we will log only when there is an error during proxy execution. To achieve this, the policies are placed within the FaultRule. No condition is specified in the FaultRule, ensuring that the policies are triggered for all errors. Placing the policies inside the FaultRule allows us to perform specific actions when an exception occurs.

API has one policy in the pre flow that is verify API key. During testing we will try to call the API without API key to produce exception.

Faultrule
Two policies are created for sending the logs to Relic. You can attach these policies to any flow and delete it so that policies can be used only in faultrule. Make sure continueOnError=”true” is set.

Js script to format the log paylod. <!– this policy allows us to execute java script code during execution of an API Proxy –>
<Javascript async=”false” continueOnError=”true” enabled=”true” timeLimit=”200″ xmlns=’http://www.sap.com/apimgmt’>
<!– contains reference to any library scripts that help the main code file –>

<!– contains the name of the main code file –>
<ResourceURL>jsc://generateLoggingPayload.js</ResourceURL>
</Javascript> var error_content = context.getVariable(“error.content”);
var proxy_name = context.getVariable(“apiproxy.name”);
var messageid = context.getVariable(“messageid”);
var response_content = context.getVariable(“response.content”);
var response_code = context.getVariable(“error.status.code”);
var request_method = context.getVariable(“request.verb”);
var state = context.getVariable(“error.state”);
var proxy_url = context.getVariable(“request.uri”);
var target_url = context.getVariable(“request.url”);
//
// Build the payload with the stringified input
var payload = {

“message”: proxy_name,
“level”: “Error”,
“error_content” : error_content,
“messageid” : messageid,
“response_content” : response_content,
“response_code” : response_code,
“method”: request_method,
“state”:state,
“proxy_url”: proxy_url,
“target_url” : target_url

};

// Set the stringified payload for New Relic
context.setVariable(“newRelicPayload”, JSON.stringify(payload));ServiceCallout to make http api all to post payload<!– this policy lets you call to an external service from your API flow –>
<ServiceCallout async=”true” continueOnError=”true” enabled=”true” xmlns=”http://www.sap.com/apimgmt”>
<!– The request that gets sent from the API proxy flow to the external service –>
<Request clearPayload=”false” variable=”newRelicRequest”>
<Set>
<Headers>
<Header name=”X-License-Key”>Your-API-Key</Header>
<Header name=”Content-Type”>application/json</Header>
</Headers>

Next we have to attach these two policies in the fault rule

Unfortunately, we cannot attach any policy to a FaultRule using the UI. To add policies to a FaultRule manually, please follow the instructions provided in this blog https://community.sap.com/t5/technology-blog-posts-by-members/sap-api-management-handling-faults-using-faultrules-and-defaultfaultrule/ba-p/13423702

We need to download proxy and unzip and edit the Proxyendpoint and Proxytargeendpoint xmls.

ProxyEndpoint

<?xml version=”1.0″ encoding=”UTF-8″ standalone=”yes”?>
<ProxyEndPoint default=”true” xmlns=”http://www.sap.com/apimgmt”>
<name>default</name>
<base_path>/nortwind/Products</base_path>
<properties/>
<routeRules>
<routeRule>
<name>default</name>
<targetEndPointName>default</targetEndPointName>
<sequence>1</sequence>
<faultRules/>
</routeRule>
</routeRules>
<faultRules>
<faultRule>
<name>LogErrors</name>
<condition></condition>
<steps>
<step>
<policy_name>generateLoggingPayload</policy_name>
<condition> </condition>
<sequence>1</sequence>
</step>
<step>
<policy_name>sendLogsRelic</policy_name>
<condition> </condition>
<sequence>2</sequence>
</step>
</steps>
</faultRule>
</faultRules>
<preFlow>
<name>PreFlow</name>
<request>
<isRequest>true</isRequest>
<steps>
<step>
<policy_name>verifyAPIKey</policy_name>
<condition></condition>
<sequence>1</sequence>
</step>
</steps>
</request>
</preFlow>
<postFlow>
<name>PostFlow</name>
</postFlow>
<conditionalFlows/>
</ProxyEndPoint>

TargetEndPoint xml configuration.

<?xml version=”1.0″ encoding=”UTF-8″ standalone=”yes”?>
<TargetEndPoint xmlns=”http://www.sap.com/apimgmt”>
<name>default</name>
<url>https://services.odata.org/northwind/northwind.svc/Productss</url>
<provider_id>NONE</provider_id>
<additionalAPIProviders/>
<isDefault>true</isDefault>
<properties/>
<faultRules>
<faultRule>
<name>LogErrors</name>
<condition></condition>
<steps>
<step>
<policy_name>sendLogsRelic</policy_name>
<condition> </condition>
<sequence>2</sequence>
</step>
<step>
<policy_name>generateLoggingPayload</policy_name>
<condition> </condition>
<sequence>1</sequence>
</step>
</steps>
</faultRule>
</faultRules>
<preFlow>
<name>PreFlow</name>
<request>
<isRequest>true</isRequest>
<steps/>
</request>
</preFlow>
<postFlow>
<name>PostFlow</name>
</postFlow>
<conditionalFlows/>
<loadBalancerConfigurations>
<isRetry>false</isRetry>
<healthMonitor>
<isEnabled>false</isEnabled>
</healthMonitor>
</loadBalancerConfigurations>
</TargetEndPoint>

If you are using mac or linux, use this command for zipping the proxy. https://docs.apigee.com/api-platform/reference/api-proxy-configuration-reference

zip my-new-proxy.zip -r .

After making the necessary changes, compress the files into a ZIP archive and re-import them into the proxy.

Testing

APIProxyEndpointError

Send reqeust without Api key to produce APIProxyend errors.

Logs

APITargetEndpointError

add wrong url to produce proxy endpoint errors.

You can try out this approach and check if this can fulfil into your needs.

Also, if you look closely at the error details, the error appears in SAP API Management and is also available on the client side. If the client system is already logging the errors, then there’s no need to perform logging in API Management, as it would be a duplication of effort. So avoid duplicate logging.

Relic has limitations regarding payload size; therefore, it’s important to consider this when sending request and response payloads

References

https://help.sap.com/docs/sap-api-management/sap-api-management/variable-references?version=Cloud

In this blog, we’ll dive into using Newrelic as an external logging server. Logging requests and responses is simple, and I’ll guide you through logging all proxy exceptions to the Newrelic server.I have used rest api call for sending the logs to new relic similar to https://community.sap.com/t5/technology-blog-posts-by-members/splunk-part-1-sap-apim-logging-monitoring/ba-p/13444151You can check here for payload format and limitations https://docs.newrelic.com/docs/logs/log-api/introduction-log-api/Newrelic trial account setupYou can create new relic account here.https://newrelic.com/Create an application keyCopy your and check the logging functionality by testing from postman. Url is different for US and EU. I am using EU url. https://docs.newrelic.com/docs/logs/log-api/introduction-log-api/Curl curl –location ‘https://log-api.eu.newrelic.com/log/v1’
–header ‘Api-Key: YourKey’
–header ‘Content-Type: application/json’
–data ‘{
“message”: “Postman Testing”,
“Proxyname”: “SAP SCN Blogs”,
“level”: “Error”
}’ Message in Relic logs. Filter the mesasges logs based on the timeline. New Relic account setup is done and next is to send the logs from api proxy. API Proxy Configuration For demonstration purposes, we will log only when there is an error during proxy execution. To achieve this, the policies are placed within the FaultRule. No condition is specified in the FaultRule, ensuring that the policies are triggered for all errors. Placing the policies inside the FaultRule allows us to perform specific actions when an exception occurs.API has one policy in the pre flow that is verify API key. During testing we will try to call the API without API key to produce exception. FaultruleTwo policies are created for sending the logs to Relic. You can attach these policies to any flow and delete it so that policies can be used only in faultrule. Make sure continueOnError=”true” is set. Js script to format the log paylod. <!– this policy allows us to execute java script code during execution of an API Proxy –>
<Javascript async=”false” continueOnError=”true” enabled=”true” timeLimit=”200″ xmlns=’http://www.sap.com/apimgmt’>
<!– contains reference to any library scripts that help the main code file –>

};

<Payload contentType=”application/json”>{newRelicPayload}</Payload>
<Verb>POST</Verb>
</Set>
</Request>
<Response>newRelicResponse</Response>
<HTTPTargetConnection>
<URL>https://log-api.eu.newrelic.com/log/v1</URL>
</HTTPTargetConnection>
</ServiceCallout>Next we have to attach these two policies in the fault ruleUnfortunately, we cannot attach any policy to a FaultRule using the UI. To add policies to a FaultRule manually, please follow the instructions provided in this blog https://community.sap.com/t5/technology-blog-posts-by-members/sap-api-management-handling-faults-using-faultrules-and-defaultfaultrule/ba-p/13423702We need to download proxy and unzip and edit the Proxyendpoint and Proxytargeendpoint xmls. ProxyEndpoint<?xml version=”1.0″ encoding=”UTF-8″ standalone=”yes”?>
<ProxyEndPoint default=”true” xmlns=”http://www.sap.com/apimgmt”>
<name>default</name>
<base_path>/nortwind/Products</base_path>
<properties/>
<routeRules>
<routeRule>
<name>default</name>
<targetEndPointName>default</targetEndPointName>
<sequence>1</sequence>
<faultRules/>
</routeRule>
</routeRules>
<faultRules>
<faultRule>
<name>LogErrors</name>
<condition></condition>
<steps>
<step>
<policy_name>generateLoggingPayload</policy_name>
<condition> </condition>
<sequence>1</sequence>
</step>
<step>
<policy_name>sendLogsRelic</policy_name>
<condition> </condition>
<sequence>2</sequence>
</step>
</steps>
</faultRule>
</faultRules>
<preFlow>
<name>PreFlow</name>
<request>
<isRequest>true</isRequest>
<steps>
<step>
<policy_name>verifyAPIKey</policy_name>
<condition></condition>
<sequence>1</sequence>
</step>
</steps>
</request>
</preFlow>
<postFlow>
<name>PostFlow</name>
</postFlow>
<conditionalFlows/>
</ProxyEndPoint>TargetEndPoint xml configuration.<?xml version=”1.0″ encoding=”UTF-8″ standalone=”yes”?>
<TargetEndPoint xmlns=”http://www.sap.com/apimgmt”>
<name>default</name>
<url>https://services.odata.org/northwind/northwind.svc/Productss</url>
<provider_id>NONE</provider_id>
<additionalAPIProviders/>
<isDefault>true</isDefault>
<properties/>
<faultRules>
<faultRule>
<name>LogErrors</name>
<condition></condition>
<steps>
<step>
<policy_name>sendLogsRelic</policy_name>
<condition> </condition>
<sequence>2</sequence>
</step>
<step>
<policy_name>generateLoggingPayload</policy_name>
<condition> </condition>
<sequence>1</sequence>
</step>
</steps>
</faultRule>
</faultRules>
<preFlow>
<name>PreFlow</name>
<request>
<isRequest>true</isRequest>
<steps/>
</request>
</preFlow>
<postFlow>
<name>PostFlow</name>
</postFlow>
<conditionalFlows/>
<loadBalancerConfigurations>
<isRetry>false</isRetry>
<healthMonitor>
<isEnabled>false</isEnabled>
</healthMonitor>
</loadBalancerConfigurations>
</TargetEndPoint> If you are using mac or linux, use this command for zipping the proxy. https://docs.apigee.com/api-platform/reference/api-proxy-configuration-referencezip my-new-proxy.zip -r .After making the necessary changes, compress the files into a ZIP archive and re-import them into the proxy.TestingAPIProxyEndpointErrorSend reqeust without Api key to produce APIProxyend errors. LogsAPITargetEndpointErroradd wrong url to produce proxy endpoint errors. You can try out this approach and check if this can fulfil into your needs. Also, if you look closely at the error details, the error appears in SAP API Management and is also available on the client side. If the client system is already logging the errors, then there’s no need to perform logging in API Management, as it would be a duplication of effort. So avoid duplicate logging. Relic has limitations regarding payload size; therefore, it’s important to consider this when sending request and response payloadsReferenceshttps://help.sap.com/docs/sap-api-management/sap-api-management/variable-references?version=Cloud Read More Technology Blog Posts by Members articles