Loading custom format data into splunk
I am new to splunk and need some clarification on the best approach to preprocess. I have a file in the following .csv format
field1, field2, field3, field4, field5
dummy dummy date(YYYYMMMDD) dummy time
The time does not have the 0 preset, so for example 13 seconds would be listed as .. '13', 1 hour 50 minutes and 22 seconds would be 15022.
Is it possible to resolve this via the default input loader via regex?. It says that 0's don't matter but the time comes out wrong, I have Y%m%d%H%M%S .
The second approach that I been looking at (if someone can point me to a quick guide people) how can I configure so for every matching *file.csv a python rule is triggered? (I don't want it to run at intervals, whenever data is being index/imported into spunk)
Thank you.
python regex splunk
asked Nov 21 '18 at 15:32
MarkMark
62
add a comment |
I am new to splunk and need some clarification on the best approach to preprocess. I have a file in the following .csv format
field1, field2, field3, field4, field5
dummy dummy date(YYYYMMMDD) dummy time
The time does not have the 0 preset, so for example 13 seconds would be listed as .. '13', 1 hour 50 minutes and 22 seconds would be 15022.
Is it possible to resolve this via the default input loader via regex?. It says that 0's don't matter but the time comes out wrong, I have Y%m%d%H%M%S .
The second approach that I been looking at (if someone can point me to a quick guide people) how can I configure so for every matching *file.csv a python rule is triggered? (I don't want it to run at intervals, whenever data is being index/imported into spunk)
Thank you.
python regex splunk
asked Nov 21 '18 at 15:32
MarkMark
62
add a comment |
I am new to splunk and need some clarification on the best approach to preprocess. I have a file in the following .csv format
field1, field2, field3, field4, field5
dummy dummy date(YYYYMMMDD) dummy time
The time does not have the 0 preset, so for example 13 seconds would be listed as .. '13', 1 hour 50 minutes and 22 seconds would be 15022.
Is it possible to resolve this via the default input loader via regex?. It says that 0's don't matter but the time comes out wrong, I have Y%m%d%H%M%S .
The second approach that I been looking at (if someone can point me to a quick guide people) how can I configure so for every matching *file.csv a python rule is triggered? (I don't want it to run at intervals, whenever data is being index/imported into spunk)
Thank you.
python regex splunk
asked Nov 21 '18 at 15:32
MarkMark
62
I am new to splunk and need some clarification on the best approach to preprocess. I have a file in the following .csv format
field1, field2, field3, field4, field5
dummy dummy date(YYYYMMMDD) dummy time
The time does not have the 0 preset, so for example 13 seconds would be listed as .. '13', 1 hour 50 minutes and 22 seconds would be 15022.
Is it possible to resolve this via the default input loader via regex?. It says that 0's don't matter but the time comes out wrong, I have Y%m%d%H%M%S .
The second approach that I been looking at (if someone can point me to a quick guide people) how can I configure so for every matching *file.csv a python rule is triggered? (I don't want it to run at intervals, whenever data is being index/imported into spunk)
Thank you.
python regex splunk
python regex splunk
asked Nov 21 '18 at 15:32
MarkMark
62
asked Nov 21 '18 at 15:32
MarkMark
62
asked Nov 21 '18 at 15:32
MarkMark
62
asked Nov 21 '18 at 15:32
MarkMark
62
asked Nov 21 '18 at 15:32
MarkMark
62
62
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
Does the time field contain time-of-day or elapsed time? Splunk cannot handle the latter.
Consider creating modular input(s) (Python scripts) to read the file and convert the fields as necessary. The output of the scripts will the indexed by Splunk.
answered Nov 21 '18 at 22:09
RichGRichG
7011410
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53415441%2floading-custom-format-data-into-splunk%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Does the time field contain time-of-day or elapsed time? Splunk cannot handle the latter.
Consider creating modular input(s) (Python scripts) to read the file and convert the fields as necessary. The output of the scripts will the indexed by Splunk.
answered Nov 21 '18 at 22:09
RichGRichG
7011410
add a comment |
Does the time field contain time-of-day or elapsed time? Splunk cannot handle the latter.
Consider creating modular input(s) (Python scripts) to read the file and convert the fields as necessary. The output of the scripts will the indexed by Splunk.
answered Nov 21 '18 at 22:09
RichGRichG
7011410
add a comment |
Does the time field contain time-of-day or elapsed time? Splunk cannot handle the latter.
Consider creating modular input(s) (Python scripts) to read the file and convert the fields as necessary. The output of the scripts will the indexed by Splunk.
answered Nov 21 '18 at 22:09
RichGRichG
7011410
Does the time field contain time-of-day or elapsed time? Splunk cannot handle the latter.
Consider creating modular input(s) (Python scripts) to read the file and convert the fields as necessary. The output of the scripts will the indexed by Splunk.
answered Nov 21 '18 at 22:09
RichGRichG
7011410
answered Nov 21 '18 at 22:09
RichGRichG
7011410
answered Nov 21 '18 at 22:09
RichGRichG
7011410
answered Nov 21 '18 at 22:09
RichGRichG
7011410
7011410
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53415441%2floading-custom-format-data-into-splunk%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
