Enhancing Multi-Model Server Logging Support - JSON Format Integration

[ViliamSerecun]

Description:

In an effort to improve the flexibility and integration capacity of our Multi-Model Server (MMS) with third-party log management solutions, such as Datadog, I propose that we extend the existing logging functionality to support JSON log format.

Currently, the MMS properties can be configured, but the logging format is restricted and does not permit users to alter the log4j2.xml file to implement JSON logging. The existing log4j2.xml configuration is as follows:

<Console name="STDOUT" target="SYSTEM_OUT">
    <PatternLayout pattern="%d{ISO8601} [%-5p] %t %c - %m%n"/>
</Console>

To enable JSON logging (for instance, in a CloudWatch environment), the configuration would need modification to something like:

<Console name="STDOUT" target="SYSTEM_OUT">
    <JSONLayout compact="true" eventEol="true" properties="true" stacktraceAsString="true" includeTimeMillis="true"/>
</Console>

This modification, however, would necessitate the installation of additional dependencies for the multi-model-server:

https://repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.12.3/jackson-databind-2.12.3.jar
https://repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.12.3/jackson-annotations-2.12.3.jar
https://repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-core/2.12.3/jackson-core-2.12.3.jar

Proposed Solution

The optimal solution would be to incorporate the XML configuration into an external configuration file and introduce a user-configurable property that allows users to toggle JSON logging on or off as needed. This can be added to the config.properties file:

json_logging=true/false

This adjustment would drastically improve the server's versatility and adaptability to various logging requirements and environments.

Alternative Solution

An additional option to consider would be the introduction of an environment variable that points to the log4j2.xml file's location. This variable could be used at the entry point when initiating the model server start process.

this adjustment could be done here

https://github.com/aws/sagemaker-inference-toolkit/blob/master/src/sagemaker_inference/model_server.py#L43