I'm trying to setup logstash in docker. I'm using the logstash:8.0.0 image.

This is my logstash.yml

http.host: "0.0.0.0"
xpack.monitoring.enabled: false

This is my pipeline.conf

input {
  beats {
    port => 5044
  }
}

output {
  elasticsearch {
    hosts => ["http://10.135.95.164:9200"]
    index => "instameister"
    username => "elastic"
    password => ""
  }
  stdout { codec => rubydebug }
}

And this is the error im getting:

Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"Java::JavaLang::IllegalStateException", :message=>"Unable to configure plugins: (ConfigurationError) Something is wrong with your configuration.", :backtrace=>["org.logstash.config.ir.CompiledPipeline.<init>(CompiledPipeline.java:120)", "org.logstash.execution.JavaBasePipelineExt.initialize(JavaBasePipelineExt.java:85)", "org.logstash.execution.JavaBasePipelineExt$INVOKER$i$1$0$initialize.call(JavaBasePipelineExt$INVOKER$i$1$0$initialize.gen)", "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:837)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuper(IRRuntimeHelpers.java:1169)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuperSplatArgs(IRRuntimeHelpers.java:1156)", "org.jruby.ir.targets.InstanceSuperInvokeSite.invoke(InstanceSuperInvokeSite.java:39)", "usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$initialize$0(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:47)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:333)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:87)", "org.jruby.RubyClass.newInstance(RubyClass.java:939)", "org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInstance.gen)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)", "usr.share.logstash.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0(/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:50)", "usr.share.logstash.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0$__VARARGS__(/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:49)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)", "usr.share.logstash.logstash_minus_core.lib.logstash.agent.RUBY$block$converge_state$2(/usr/share/logstash/logstash-core/lib/logstash/agent.rb:376)", "org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:138)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:52)", "org.jruby.runtime.Block.call(Block.java:139)", "org.jruby.RubyProc.call(RubyProc.java:318)", "org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)", "java.base/java.lang.Thread.run(Thread.java:829)"]}

All i see is the Unable to configure plugins: (ConfigurationError) Something is wrong with your configuration. But i have no idea whats wrong.

CodePudding user response：

Instead of modifying logstash.yml, you can override the variables in the environment instead. Your pipeline.conf seems to be Ok. rubydebug codec is enabled by default for stdout.

So, assuming that you have a docker compose file, the configuration would be something like this:

  logstash:
    image: docker.elastic.co/logstash/logstash
    container_name: logstash
    restart: always
    user: root
    volumes:
      - ./logstash/pipeline:/usr/share/logstash/pipeline:ro
      - ./logstash/logs/:/logstash/logs/:rw
    environment:
      - xpack.monitoring.enabled=false 
      - outputs.elasticsearch=http://elasticuser:elasticuserpassword@elasticsearch:9200
    depends_on:
      - elasticsearch

In the ./logstash/pipeline directory, a logstash.conf file with:

input {
  beats {
    port => 5044
  }
}

output {
  elasticsearch {
    hosts => "${outputs.elasticsearch}"
  }

  stdout {
  }
}

Adapt to your needs.

CodePudding user response：

The problem was that the key 'username' should be 'user' This is the working config:

input {
  beats {
    port => 5044
  }
}

output {
  elasticsearch {
    hosts => ["http://10.135.95.164:9200"]
    user => "elastic"
    password => ""
    index => "instameister"
    manage_template => false
  }
  stdout { codec => json_lines }
}