Skip to content

Conversation

@SylvainJuge
Copy link
Member

@SylvainJuge SylvainJuge commented Mar 14, 2022

What does this PR do?

Issue a written warning in the process standard error output as there is nothing that the agent can do:

[elastic-apm-agent] WARN Security manager without agent grant-all permission, adding the following snippet to security policy is recommended: [elastic-apm-agent] WARN grant codeBase "file:/path/to/elastic-apm-agent.jar" { [elastic-apm-agent] WARN permission java.security.AllPermission; [elastic-apm-agent] WARN }; 

We have to do this very early in the agent startup as we can't even check for defined properties or use agent configuration (which would have been convenient to use the actual path to the java agent).

Checklist

  • This is something else
    • I have updated CHANGELOG.asciidoc
    • Testing manually with Tomcat 9.0.x ./bin/catalina.sh run -security + policy in ./conf/catalina.policy
    • While the added check is working properly, there is still some issues with Tomcat+SM enabled, but this might be related to a general incompatibility between our classloading strategy and SM and/or Tomcat (see stack trace below). The same issue happens with agent 1.29.0, thus it's not a regression.
Thrown exception with Tomcat + security manager
java.lang.reflect.InvocationTargetException at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at java.base/java.lang.IndyBootstrapDispatcher.bootstrap(IndyBootstrapDispatcher.java:60) at java.base/java.lang.invoke.BootstrapMethodInvoker.invoke(BootstrapMethodInvoker.java:138) at java.base/java.lang.invoke.CallSite.makeSite(CallSite.java:307) at java.base/java.lang.invoke.MethodHandleNatives.linkCallSiteImpl(MethodHandleNatives.java:258) at java.base/java.lang.invoke.MethodHandleNatives.linkCallSite(MethodHandleNatives.java:248) at co.elastic.apm.agent.bci.bytebuddy.Instrumented.isInstrumented(Instrumented.java:34) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at co.elastic.apm.agent.bci.bytebuddy.InstallationListenerImpl.onAfterWarmUp(InstallationListenerImpl.java:65) at net.bytebuddy.agent.builder.AgentBuilder$InstallationListener$Compound.onAfterWarmUp(AgentBuilder.java:5221) at net.bytebuddy.agent.builder.AgentBuilder$Default$WarmupStrategy$Enabled.apply(AgentBuilder.java:11087) at net.bytebuddy.agent.builder.AgentBuilder$Default.doInstall(AgentBuilder.java:10774) at net.bytebuddy.agent.builder.AgentBuilder$Default.installOn(AgentBuilder.java:10712) at co.elastic.apm.agent.bci.ElasticApmAgent.initInstrumentation(ElasticApmAgent.java:279) at co.elastic.apm.agent.bci.ElasticApmAgent.initInstrumentation(ElasticApmAgent.java:164) at co.elastic.apm.agent.bci.ElasticApmAgent.initialize(ElasticApmAgent.java:150) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at co.elastic.apm.agent.premain.AgentMain.loadAndInitializeAgent(AgentMain.java:160) at co.elastic.apm.agent.premain.AgentMain.init(AgentMain.java:101) at co.elastic.apm.agent.premain.AgentMain.premain(AgentMain.java:50) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:513) at java.instrument/sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:525) Caused by: java.lang.NoClassDefFoundError: org/apache/logging/log4j/message/ReusableSimpleMessage at org.apache.logging.log4j.message.ReusableMessageFactory.getSimple(ReusableMessageFactory.java:64) at org.apache.logging.log4j.message.ReusableMessageFactory.newMessage(ReusableMessageFactory.java:180) at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2003) at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1975) at co.elastic.apm.agent.logging.Log4j2LoggerBridge.error(Log4j2LoggerBridge.java:192) at co.elastic.apm.agent.bci.IndyBootstrap.bootstrap(IndyBootstrap.java:412) ... 35 more Caused by: java.lang.ClassNotFoundException: org.apache.logging.log4j.message.ReusableSimpleMessage at co.elastic.apm.agent.premain.ShadedClassLoader.findClass(ShadedClassLoader.java:77) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) ... 41 more 
@ghost
Copy link

ghost commented Mar 14, 2022

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-03-21T08:23:36.725+0000

  • Duration: 47 min 14 sec

Test stats 🧪

Test Results
Failed 0
Passed 2749
Skipped 20
Total 2769

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • run benchmark tests : Run the benchmark tests.

  • run jdk compatibility tests : Run the JDK Compatibility tests.

  • run integration tests : Run the Agent Integration tests.

  • run end-to-end tests : Run the APM-ITs.

  • run windows tests : Build & tests on windows.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@eyalkoren
Copy link
Contributor

Nice!
Regarding this error, it happens during BB warmup, but not sure it is related.
The easiest way to check is start the agent with warmup_byte_buddy=false (see https://github.com/elastic/apm-agent-java/blob/main/apm-agent-core/src/main/java/co/elastic/apm/agent/bci/ElasticApmAgent.java#L272-L276).
If it's related, we can initialize our logging only after warmup, so that our SDK noop logger will be used during warmup and the real logger will be used later.
If not specific to warmup, debugging this will probably tell you why that is.

@SylvainJuge
Copy link
Member Author

The thrown exception is still there even when trying with warmup_byte_buddy=false, thus I've opened #2520 so we can deal with this later.

@SylvainJuge SylvainJuge marked this pull request as ready for review March 17, 2022 09:44
@github-actions
Copy link

/test

return;
}
try {
sm.checkPermission(new AllPermission());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if someone has configured more fine-grained permissions, such as

 permission java.lang.RuntimePermission "setFactory"; permission java.lang.RuntimePermission "getClassLoader"; permission java.lang.RuntimePermission "setContextClassLoader"; permission java.net.SocketPermission "*", "connect,resolve"; 
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think here we can rely on the past experience of @eyalkoren with that: there are simply lots of them.
Even simple things like reading a system property need to be explicitly enabled when using fine-grained permissions, and some like SocketPermission might have their values depend on server_url in agent configuration that can't be read and thus be added to the policy snippet here.

If the need to provide fine-grained details arises, then we can do the extra effort of gathering (and maintaining) an exhaustive list.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trying to maintain a minimal set of permissions sounds like a terrible idea for me. Even if we compile such, any agent upgrade may break due to additional required permission.
I am personally fine with issuing this warning if AllPermission is not granted. If the user does maintain a proper policy with minimal permissions, the agent will work as expected and this warning can be ignored. On the other hand, if we just check specific permission here (e.g. try to getClassLoader or read a system property) and the permission set in the policy is not exhaustive, there won't be a warning and the agent will not work as expected, which is worse.

return;
}
try {
sm.checkPermission(new AllPermission());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trying to maintain a minimal set of permissions sounds like a terrible idea for me. Even if we compile such, any agent upgrade may break due to additional required permission.
I am personally fine with issuing this warning if AllPermission is not granted. If the user does maintain a proper policy with minimal permissions, the agent will work as expected and this warning can be ignored. On the other hand, if we just check specific permission here (e.g. try to getClassLoader or read a system property) and the permission set in the policy is not exhaustive, there won't be a warning and the agent will not work as expected, which is worse.

@SylvainJuge SylvainJuge enabled auto-merge (squash) March 21, 2022 08:24
@SylvainJuge SylvainJuge merged commit c323326 into elastic:main Mar 21, 2022
@SylvainJuge SylvainJuge deleted the security-manager-warn branch March 21, 2022 10:13
@felixbarny
Copy link
Member

Could this have been implemented as a BootstrapCheck, or are these ran too late in the attachment process?

@SylvainJuge
Copy link
Member Author

Could this have been implemented as a BootstrapCheck, or are these ran too late in the attachment process?

Yes, I initially thought of that, but by default we can't even read a system property, as a result we can't have a good hint for configuration:

  • we can't provide the actual path to the agent Jar
  • we can't provide a link to the policy file that is used for the "simple case" where a single policy file is used.
@SylvainJuge SylvainJuge mentioned this pull request Mar 23, 2022
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

3 participants