Is it possible to configure secure(SSL) kafka with Telegraf?
Started working with TICK stack, I'm quite new to this. A dedicated machine is assigned for TICK to run and receive data. To begin with I have sent metrics from 3 instances to TICK stack using udp and everything is good. Now, I would like to send the Kafka information too, which is a secure(ssl-certified). Secure certificates came with files: ca-cert, ca-cert.srl, ca-key, cert-file, cert-signed, kafka.client.truststore.jks, kafka.server.keystore.jks, kafka.server.truststore.jks.
On the other end, I'm trying to configure telegraf.conf file with TSL config parameters for
tls_ca = "ca-cert"
tls_cert = "cert-signed"
tls_key = "ca-key"
On starting telegraf the error shown is:
2019-01-18T11:51:48Z E! [agent] Failed to connect to output kafka, retrying in 15s, error was 'could not load keypair cert-signed:ca-key: tls: failed to parse private key'
2019-01-18T11:52:03Z E! [telegraf] Error running agent: could not load keypair cert-signed:ca-key: tls: failed to parse private key
Not sure what's going on. Help me debug or pick right files.
I understand that kafka.client.truststore.jks is used while any of the client try to access kafka.
But how is that used here, being telegraf is a client trying to publish?
ssl apache-kafka ssl-certificate telegraf telegraf-output-plugins
edited Jan 18 at 12:31
emix
6,36542948
asked Jan 18 at 12:24
srikanthsrikanth
459521
add a comment |
Started working with TICK stack, I'm quite new to this. A dedicated machine is assigned for TICK to run and receive data. To begin with I have sent metrics from 3 instances to TICK stack using udp and everything is good. Now, I would like to send the Kafka information too, which is a secure(ssl-certified). Secure certificates came with files: ca-cert, ca-cert.srl, ca-key, cert-file, cert-signed, kafka.client.truststore.jks, kafka.server.keystore.jks, kafka.server.truststore.jks.
On the other end, I'm trying to configure telegraf.conf file with TSL config parameters for
tls_ca = "ca-cert"
tls_cert = "cert-signed"
tls_key = "ca-key"
On starting telegraf the error shown is:
2019-01-18T11:51:48Z E! [agent] Failed to connect to output kafka, retrying in 15s, error was 'could not load keypair cert-signed:ca-key: tls: failed to parse private key'
2019-01-18T11:52:03Z E! [telegraf] Error running agent: could not load keypair cert-signed:ca-key: tls: failed to parse private key
Not sure what's going on. Help me debug or pick right files.
I understand that kafka.client.truststore.jks is used while any of the client try to access kafka.
But how is that used here, being telegraf is a client trying to publish?
ssl apache-kafka ssl-certificate telegraf telegraf-output-plugins
edited Jan 18 at 12:31
emix
6,36542948
asked Jan 18 at 12:24
srikanthsrikanth
459521
JKS files are for the Java keystore. Since Telegraf doesn't use the Java clients, probably not needed
– cricket_007
Jan 18 at 20:16
@cricket_007 not required is not valid i believe. While the Kafka instance is secure, producer/consumer would not allow the client to communicate with the server with out the encryption/decryption policy.
– srikanth
yesterday
1
Telegraf uses Sarama client. github.com/influxdata/telegraf/blob/master/plugins/outputs/… And I don't see JKS being used or configured here medium.com/processone/…
– cricket_007
22 hours ago
Your error says it failed to parse the keys, so you should show how you generated them. But it is possible, based on this config section github.com/influxdata/telegraf/blob/master/plugins/outputs/…
– cricket_007
22 hours ago
add a comment |
Started working with TICK stack, I'm quite new to this. A dedicated machine is assigned for TICK to run and receive data. To begin with I have sent metrics from 3 instances to TICK stack using udp and everything is good. Now, I would like to send the Kafka information too, which is a secure(ssl-certified). Secure certificates came with files: ca-cert, ca-cert.srl, ca-key, cert-file, cert-signed, kafka.client.truststore.jks, kafka.server.keystore.jks, kafka.server.truststore.jks.
On the other end, I'm trying to configure telegraf.conf file with TSL config parameters for
tls_ca = "ca-cert"
tls_cert = "cert-signed"
tls_key = "ca-key"
On starting telegraf the error shown is:
2019-01-18T11:51:48Z E! [agent] Failed to connect to output kafka, retrying in 15s, error was 'could not load keypair cert-signed:ca-key: tls: failed to parse private key'
2019-01-18T11:52:03Z E! [telegraf] Error running agent: could not load keypair cert-signed:ca-key: tls: failed to parse private key
Not sure what's going on. Help me debug or pick right files.
I understand that kafka.client.truststore.jks is used while any of the client try to access kafka.
But how is that used here, being telegraf is a client trying to publish?
ssl apache-kafka ssl-certificate telegraf telegraf-output-plugins
edited Jan 18 at 12:31
emix
6,36542948
asked Jan 18 at 12:24
srikanthsrikanth
459521
Started working with TICK stack, I'm quite new to this. A dedicated machine is assigned for TICK to run and receive data. To begin with I have sent metrics from 3 instances to TICK stack using udp and everything is good. Now, I would like to send the Kafka information too, which is a secure(ssl-certified). Secure certificates came with files: ca-cert, ca-cert.srl, ca-key, cert-file, cert-signed, kafka.client.truststore.jks, kafka.server.keystore.jks, kafka.server.truststore.jks.
On the other end, I'm trying to configure telegraf.conf file with TSL config parameters for
tls_ca = "ca-cert"
tls_cert = "cert-signed"
tls_key = "ca-key"
On starting telegraf the error shown is:
2019-01-18T11:51:48Z E! [agent] Failed to connect to output kafka, retrying in 15s, error was 'could not load keypair cert-signed:ca-key: tls: failed to parse private key'
2019-01-18T11:52:03Z E! [telegraf] Error running agent: could not load keypair cert-signed:ca-key: tls: failed to parse private key
Not sure what's going on. Help me debug or pick right files.
I understand that kafka.client.truststore.jks is used while any of the client try to access kafka.
But how is that used here, being telegraf is a client trying to publish?
ssl apache-kafka ssl-certificate telegraf telegraf-output-plugins
ssl apache-kafka ssl-certificate telegraf telegraf-output-plugins
edited Jan 18 at 12:31
emix
6,36542948
asked Jan 18 at 12:24
srikanthsrikanth
459521
edited Jan 18 at 12:31
emix
6,36542948
asked Jan 18 at 12:24
srikanthsrikanth
459521
edited Jan 18 at 12:31
emix
6,36542948
edited Jan 18 at 12:31
emix
6,36542948
edited Jan 18 at 12:31
emix
6,36542948
6,36542948
asked Jan 18 at 12:24
srikanthsrikanth
459521
asked Jan 18 at 12:24
srikanthsrikanth
459521
asked Jan 18 at 12:24
srikanthsrikanth
459521
459521
JKS files are for the Java keystore. Since Telegraf doesn't use the Java clients, probably not needed
– cricket_007
Jan 18 at 20:16
@cricket_007 not required is not valid i believe. While the Kafka instance is secure, producer/consumer would not allow the client to communicate with the server with out the encryption/decryption policy.
– srikanth
yesterday
1
Telegraf uses Sarama client. github.com/influxdata/telegraf/blob/master/plugins/outputs/… And I don't see JKS being used or configured here medium.com/processone/…
– cricket_007
22 hours ago
Your error says it failed to parse the keys, so you should show how you generated them. But it is possible, based on this config section github.com/influxdata/telegraf/blob/master/plugins/outputs/…
– cricket_007
22 hours ago
add a comment |
JKS files are for the Java keystore. Since Telegraf doesn't use the Java clients, probably not needed
– cricket_007
Jan 18 at 20:16
@cricket_007 not required is not valid i believe. While the Kafka instance is secure, producer/consumer would not allow the client to communicate with the server with out the encryption/decryption policy.
– srikanth
yesterday
1
Telegraf uses Sarama client. github.com/influxdata/telegraf/blob/master/plugins/outputs/… And I don't see JKS being used or configured here medium.com/processone/…
– cricket_007
22 hours ago
Your error says it failed to parse the keys, so you should show how you generated them. But it is possible, based on this config section github.com/influxdata/telegraf/blob/master/plugins/outputs/…
– cricket_007
22 hours ago
JKS files are for the Java keystore. Since Telegraf doesn't use the Java clients, probably not needed
– cricket_007
Jan 18 at 20:16
JKS files are for the Java keystore. Since Telegraf doesn't use the Java clients, probably not needed
– cricket_007
Jan 18 at 20:16
@cricket_007 not required is not valid i believe. While the Kafka instance is secure, producer/consumer would not allow the client to communicate with the server with out the encryption/decryption policy.
– srikanth
yesterday
@cricket_007 not required is not valid i believe. While the Kafka instance is secure, producer/consumer would not allow the client to communicate with the server with out the encryption/decryption policy.
– srikanth
yesterday
1
1
Telegraf uses Sarama client. github.com/influxdata/telegraf/blob/master/plugins/outputs/… And I don't see JKS being used or configured here medium.com/processone/…
– cricket_007
22 hours ago
Telegraf uses Sarama client. github.com/influxdata/telegraf/blob/master/plugins/outputs/… And I don't see JKS being used or configured here medium.com/processone/…
– cricket_007
22 hours ago
Your error says it failed to parse the keys, so you should show how you generated them. But it is possible, based on this config section github.com/influxdata/telegraf/blob/master/plugins/outputs/…
– cricket_007
22 hours ago
Your error says it failed to parse the keys, so you should show how you generated them. But it is possible, based on this config section github.com/influxdata/telegraf/blob/master/plugins/outputs/…
– cricket_007
22 hours ago
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54253979%2fis-it-possible-to-configure-securessl-kafka-with-telegraf%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54253979%2fis-it-possible-to-configure-securessl-kafka-with-telegraf%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
JKS files are for the Java keystore. Since Telegraf doesn't use the Java clients, probably not needed
– cricket_007
Jan 18 at 20:16
@cricket_007 not required is not valid i believe. While the Kafka instance is secure, producer/consumer would not allow the client to communicate with the server with out the encryption/decryption policy.
– srikanth
yesterday
1
Telegraf uses Sarama client. github.com/influxdata/telegraf/blob/master/plugins/outputs/… And I don't see JKS being used or configured here medium.com/processone/…
– cricket_007
22 hours ago
Your error says it failed to parse the keys, so you should show how you generated them. But it is possible, based on this config section github.com/influxdata/telegraf/blob/master/plugins/outputs/…
– cricket_007
22 hours ago