fix: adjusting container instrumentation infrastructure monitoring to highlight assignment of container logs #16709
Conversation
…light Signed-off-by: Harry Kimpel <[email protected]>
… highlight Signed-off-by: Harry Kimpel <[email protected]>
|
|
|
|
|
Hi @harrykimpel 👋 Thanks for your pull request! Your PR is in a queue, and a writer will take a look soon. We generally publish small edits within one business day, and larger edits within three days. We will automatically generate a preview of your request, and will comment with a link when the preview is ready (usually 10 to 20 minutes). If you add any more commits, you can comment |
✅ Deploy Preview for docs-website-netlify ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
harrykimpel
changed the title
akristen
added
content
…r docker compose Signed-off-by: Harry Kimpel <[email protected]>
|
Taking a look at this today! |
This commit does not include updating pathing to add log parsing rules in the UI. I am checking if it will build remotely
|
|
||
| Our [infrastructure agent](/docs/infrastructure/infrastructure-monitoring/get-started/get-started-infrastructure-monitoring) can gather logs from your containers. When you enable logs from the infrastructure agent, the agent will automatically assign those logs to the hosts running Docker Compose. | ||
|
|
||
| ## Requirements [#requirements] |
There was a problem hiding this comment.
Do customers need to have instrumented their containers? Should we instead link them to the other containers dock as a prerequisite? I imagine they can't collect logs about containers w/o having instrumentation first.
There was a problem hiding this comment.
@akristen yes, that is correct. I guess it makes sense to link them to the other containers doc.
...rastructure-agent/linux-installation/container-instrumentation-infrastructure-monitoring.mdx
Outdated
Show resolved
Hide resolved
|
leaving a note for myself to update the URL in i18n (or revert that change) once i get SME approval from Core team |
...ructure/install-infrastructure-agent/linux-installation/enable-logs-monitored-containers.mdx
Outdated
Show resolved
Hide resolved
… highlight assignment of container log Signed-off-by: Harry Kimpel <[email protected]>
There was a problem hiding this comment.
Hey @harrykimpel, I think there could be a safer (and easier) alternative here, to ensure that logs are cleanly associated with a container and they do not get associated with a host. We could modify the Fluent Bit configuration generated by the Infra-Agent so that it removes the entity.guid.INFRA of the host if it detects that it comes from a Docker container, and then just renames attrs.tag into containerName. I think we could (and should) implement this as code, and do it out-of-the-box in the Infra-Agent. Please let me discuss this internally and will get back to you. For the time being, I'd like to ask this to wait a little bit before getting merged.
...ructure/install-infrastructure-agent/linux-installation/enable-logs-monitored-containers.mdx
Outdated
Show resolved
Hide resolved
| <img | ||
| title="Add the containers path to your snippet" | ||
| alt="A screenshot that shows where to add this snippet in context of your yaml file" | ||
| src={infrastructureAddContainersLogPath} | ||
| /> | ||
|
|
||
| * You may find this file at this path: `/etc/newrelic-infra/logging.d/logging.yml`. | ||
| * This section directs the agent to collect logs from your containers |
There was a problem hiding this comment.
We probably don't need to add this. Please note that the Log Forwarder integration picks any YAML file under /etc/newrelic-infra/logging.d/*.yml and translates them to Fluent Bit configuration. Contrarily to the public belief, they don't need to be placed under a single /etc/newrelic-infra/logging.d/logging.yml file. I believe this missconception was caused by Virtuoso, because that is the file it is using.
In this case, I think it makes sense to just use a separate file to include the Docker container logs config. It results in simpler installation instructions as well, IMHO, as we don't need to tell the user "edit the file in this particular way" (error prone, specially when editing YAML files), but just "create a file with these exact contents".
...ructure/install-infrastructure-agent/linux-installation/enable-logs-monitored-containers.mdx
Outdated
Show resolved
Hide resolved
|
|
||
| </figcaption> | ||
|
|
||
| 5. Generate some data, wait a few minutes, and you'll see container logs assigned to your containers running on your host, rather than assigned to the host itself. |
There was a problem hiding this comment.
I'm confused here. The Infra-Agent log forwarder normally gets the entity.guid belonging to the host running these containers, so the logs get associated to the host. Are you doing something differently in this case, when running Docker containers? My understanding is that by having containerName in the logs, you are somehow leveraging some Entity Synthesis rule to associate the log to a particular container. And I assume this is happening because the Infra-Agent actually sends entity.guid.INFRA, and not entity.guid. So by the time entity synthesis runs, entity.guid is still not populated.
I'm afraid that this solution is making some assumptions in the order in which the backend will perform the actions (that is, first it will execute parsing, then entity synthesis, and later on entity.guid consolidation), so I'd probably advise against this. I'd not like to have this coupling in the pipeline, as it may block us from being able to evolve it and modify the step order if we ever need to.
PS: have you verified that if logs get associated to the container they don't get associated to the host as well?
|
I'm adding this doc to sprint starting 29 APR |
|
Closing this PR until @jsubirat 's team notifies us with a better solution to complete this action. |
Please follow conventional commit standards
in your commit messages and pull request title.
Give us some context
What problems does this PR solve?
This PR will highlight some required changes in order to assign container logs to our container entities rather than the host entity
Add any context that will help us review your changes such as testing notes,
links to related docs, screenshots, etc.
there are some threads in our forum (see here and here)
If your issue relates to an existing GitHub issue, please link to it.
n/a