what is the issue with rapidminer hbos process ?
I have a dataset of more than 1 million transactions running on windows with 32 GB RAM with HBOS algorithm.
The issue is we are getting an Out of Memory Error.
Can anyone help.
rapidminer
asked Nov 23 '18 at 12:44
saurabh sawantsaurabh sawant
85
add a comment |
I have a dataset of more than 1 million transactions running on windows with 32 GB RAM with HBOS algorithm.
The issue is we are getting an Out of Memory Error.
Can anyone help.
rapidminer
asked Nov 23 '18 at 12:44
saurabh sawantsaurabh sawant
85
Hi, could you please give some more information about your data and the settings of the operator? I guess you are using the "Histogram-based Outlier Score (HBOS)" operator from the Anomaly Detection extension?
– David
Nov 26 '18 at 8:58
Hi David, the data consist's of features as follows: customer_id,msg_id,otherparty_id,sendingbank_id,amt,recevingbank_id,year,quarter,month,date_bin,day_bin, hour_bin with HBOS operator with parameter : parameter mode "all",no. of bins "-1", select mode "fixed bandwith"
– saurabh sawant
Dec 4 '18 at 9:27
add a comment |
I have a dataset of more than 1 million transactions running on windows with 32 GB RAM with HBOS algorithm.
The issue is we are getting an Out of Memory Error.
Can anyone help.
rapidminer
asked Nov 23 '18 at 12:44
saurabh sawantsaurabh sawant
85
I have a dataset of more than 1 million transactions running on windows with 32 GB RAM with HBOS algorithm.
The issue is we are getting an Out of Memory Error.
Can anyone help.
rapidminer
rapidminer
asked Nov 23 '18 at 12:44
saurabh sawantsaurabh sawant
85
asked Nov 23 '18 at 12:44
saurabh sawantsaurabh sawant
85
asked Nov 23 '18 at 12:44
saurabh sawantsaurabh sawant
85
asked Nov 23 '18 at 12:44
saurabh sawantsaurabh sawant
85
asked Nov 23 '18 at 12:44
saurabh sawantsaurabh sawant
85
85
Hi, could you please give some more information about your data and the settings of the operator? I guess you are using the "Histogram-based Outlier Score (HBOS)" operator from the Anomaly Detection extension?
– David
Nov 26 '18 at 8:58
Hi David, the data consist's of features as follows: customer_id,msg_id,otherparty_id,sendingbank_id,amt,recevingbank_id,year,quarter,month,date_bin,day_bin, hour_bin with HBOS operator with parameter : parameter mode "all",no. of bins "-1", select mode "fixed bandwith"
– saurabh sawant
Dec 4 '18 at 9:27
add a comment |
Hi, could you please give some more information about your data and the settings of the operator? I guess you are using the "Histogram-based Outlier Score (HBOS)" operator from the Anomaly Detection extension?
– David
Nov 26 '18 at 8:58
Hi David, the data consist's of features as follows: customer_id,msg_id,otherparty_id,sendingbank_id,amt,recevingbank_id,year,quarter,month,date_bin,day_bin, hour_bin with HBOS operator with parameter : parameter mode "all",no. of bins "-1", select mode "fixed bandwith"
– saurabh sawant
Dec 4 '18 at 9:27
Hi, could you please give some more information about your data and the settings of the operator? I guess you are using the "Histogram-based Outlier Score (HBOS)" operator from the Anomaly Detection extension?
– David
Nov 26 '18 at 8:58
Hi, could you please give some more information about your data and the settings of the operator? I guess you are using the "Histogram-based Outlier Score (HBOS)" operator from the Anomaly Detection extension?
– David
Nov 26 '18 at 8:58
Hi David, the data consist's of features as follows: customer_id,msg_id,otherparty_id,sendingbank_id,amt,recevingbank_id,year,quarter,month,date_bin,day_bin, hour_bin with HBOS operator with parameter : parameter mode "all",no. of bins "-1", select mode "fixed bandwith"
– saurabh sawant
Dec 4 '18 at 9:27
Hi David, the data consist's of features as follows: customer_id,msg_id,otherparty_id,sendingbank_id,amt,recevingbank_id,year,quarter,month,date_bin,day_bin, hour_bin with HBOS operator with parameter : parameter mode "all",no. of bins "-1", select mode "fixed bandwith"
– saurabh sawant
Dec 4 '18 at 9:27
add a comment |
1 Answer
1
active
oldest
votes
the HBOS algorithm can be quite memory intensive and the memory needed grows with the number of attributes used. So first of all, reducing the number of attributes might help.
But I couldn't reproduce your error. Perhaps you should reduce the max memory used by RapidMiner (under Settings -> Preferences -> System). The JVM always needs a slight overhead, so running RapidMiner with 30GB max memory should be safe.
answered Dec 7 '18 at 9:37
DavidDavid
500310
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53446954%2fwhat-is-the-issue-with-rapidminer-hbos-process%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
the HBOS algorithm can be quite memory intensive and the memory needed grows with the number of attributes used. So first of all, reducing the number of attributes might help.
But I couldn't reproduce your error. Perhaps you should reduce the max memory used by RapidMiner (under Settings -> Preferences -> System). The JVM always needs a slight overhead, so running RapidMiner with 30GB max memory should be safe.
answered Dec 7 '18 at 9:37
DavidDavid
500310
add a comment |
the HBOS algorithm can be quite memory intensive and the memory needed grows with the number of attributes used. So first of all, reducing the number of attributes might help.
But I couldn't reproduce your error. Perhaps you should reduce the max memory used by RapidMiner (under Settings -> Preferences -> System). The JVM always needs a slight overhead, so running RapidMiner with 30GB max memory should be safe.
answered Dec 7 '18 at 9:37
DavidDavid
500310
add a comment |
the HBOS algorithm can be quite memory intensive and the memory needed grows with the number of attributes used. So first of all, reducing the number of attributes might help.
But I couldn't reproduce your error. Perhaps you should reduce the max memory used by RapidMiner (under Settings -> Preferences -> System). The JVM always needs a slight overhead, so running RapidMiner with 30GB max memory should be safe.
answered Dec 7 '18 at 9:37
DavidDavid
500310
the HBOS algorithm can be quite memory intensive and the memory needed grows with the number of attributes used. So first of all, reducing the number of attributes might help.
But I couldn't reproduce your error. Perhaps you should reduce the max memory used by RapidMiner (under Settings -> Preferences -> System). The JVM always needs a slight overhead, so running RapidMiner with 30GB max memory should be safe.
answered Dec 7 '18 at 9:37
DavidDavid
500310
answered Dec 7 '18 at 9:37
DavidDavid
500310
answered Dec 7 '18 at 9:37
DavidDavid
500310
answered Dec 7 '18 at 9:37
DavidDavid
500310
500310
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53446954%2fwhat-is-the-issue-with-rapidminer-hbos-process%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Hi, could you please give some more information about your data and the settings of the operator? I guess you are using the "Histogram-based Outlier Score (HBOS)" operator from the Anomaly Detection extension?
– David
Nov 26 '18 at 8:58
Hi David, the data consist's of features as follows: customer_id,msg_id,otherparty_id,sendingbank_id,amt,recevingbank_id,year,quarter,month,date_bin,day_bin, hour_bin with HBOS operator with parameter : parameter mode "all",no. of bins "-1", select mode "fixed bandwith"
– saurabh sawant
Dec 4 '18 at 9:27