Feature (Covariates) selection in CoxPHFitter, Lifelines Survival Analysis
i am using this implemented model in Python for the purpose of survival analysis:
from lifelines import CoxPHFitter
Unfortunately i am not able(i do not know how) to loop over all covariates (features) to run the regression individualy for the purpose of feature selection and save their result. I am trying the script below:
`def fit_and_score_features2(X):
y=X[["Status","duration_yrs"]]
X.drop(["duration_yrs", "Status"], axis=1, inplace=True)
n_features = X.shape[1]
scores = np.empty(n_features)
m = CoxPHFitter()
for j in range(n_features):
Xj = X.values[:, j:j+1]
Xj=pd.merge(X, y, how='right', left_index=True, right_index=True)
m.fit(Xj, duration_col="duration_yrs", event_col="Status", show_progress=True)
scores[j] = m._score_
return scores`
Unfortunately it return me this error:
ValueError Traceback (most recent call
last) in ()
1 #Trying the function above
----> 2 scores = fit_and_score_features2(sample)
3 pd.Series(scores, index=features.columns).sort_values(ascending=False)
in fit_and_score_features2(X)
15 Xj=pd.merge(X, y, how='right', left_index=True, right_index=True)
16 m.fit(Xj, duration_col="duration_yrs", event_col="Status", show_progress=True)
---> 17 scores[j] = m.score
18 return scores
ValueError: setting an array element with a sequence.
Thank you in advance.
python feature-selection survival-analysis cox-regression lifelines
asked Nov 25 '18 at 12:17
Antonio DichevAntonio Dichev
213
add a comment |
i am using this implemented model in Python for the purpose of survival analysis:
from lifelines import CoxPHFitter
Unfortunately i am not able(i do not know how) to loop over all covariates (features) to run the regression individualy for the purpose of feature selection and save their result. I am trying the script below:
`def fit_and_score_features2(X):
y=X[["Status","duration_yrs"]]
X.drop(["duration_yrs", "Status"], axis=1, inplace=True)
n_features = X.shape[1]
scores = np.empty(n_features)
m = CoxPHFitter()
for j in range(n_features):
Xj = X.values[:, j:j+1]
Xj=pd.merge(X, y, how='right', left_index=True, right_index=True)
m.fit(Xj, duration_col="duration_yrs", event_col="Status", show_progress=True)
scores[j] = m._score_
return scores`
Unfortunately it return me this error:
ValueError Traceback (most recent call
last) in ()
1 #Trying the function above
----> 2 scores = fit_and_score_features2(sample)
3 pd.Series(scores, index=features.columns).sort_values(ascending=False)
in fit_and_score_features2(X)
15 Xj=pd.merge(X, y, how='right', left_index=True, right_index=True)
16 m.fit(Xj, duration_col="duration_yrs", event_col="Status", show_progress=True)
---> 17 scores[j] = m.score
18 return scores
ValueError: setting an array element with a sequence.
Thank you in advance.
python feature-selection survival-analysis cox-regression lifelines
asked Nov 25 '18 at 12:17
Antonio DichevAntonio Dichev
213
Why are you using_score_- that's a hidden variable, and it does not represent any kind of accuracy performance?score_however is a measure of accuracy.
– Cam.Davidson.Pilon
Nov 25 '18 at 15:58
Oh, yes you are right, but it still does not work properly. The algorithm doesn't save individual values for each variable. Return of function: X1 0.523545 X2 0.523545 X3 0.523545 X4 0.52354
– Antonio Dichev
Nov 25 '18 at 16:16
I think i was able to debug it properly
– Antonio Dichev
Nov 25 '18 at 16:26
add a comment |
i am using this implemented model in Python for the purpose of survival analysis:
from lifelines import CoxPHFitter
Unfortunately i am not able(i do not know how) to loop over all covariates (features) to run the regression individualy for the purpose of feature selection and save their result. I am trying the script below:
`def fit_and_score_features2(X):
y=X[["Status","duration_yrs"]]
X.drop(["duration_yrs", "Status"], axis=1, inplace=True)
n_features = X.shape[1]
scores = np.empty(n_features)
m = CoxPHFitter()
for j in range(n_features):
Xj = X.values[:, j:j+1]
Xj=pd.merge(X, y, how='right', left_index=True, right_index=True)
m.fit(Xj, duration_col="duration_yrs", event_col="Status", show_progress=True)
scores[j] = m._score_
return scores`
Unfortunately it return me this error:
ValueError Traceback (most recent call
last) in ()
1 #Trying the function above
----> 2 scores = fit_and_score_features2(sample)
3 pd.Series(scores, index=features.columns).sort_values(ascending=False)
in fit_and_score_features2(X)
15 Xj=pd.merge(X, y, how='right', left_index=True, right_index=True)
16 m.fit(Xj, duration_col="duration_yrs", event_col="Status", show_progress=True)
---> 17 scores[j] = m.score
18 return scores
ValueError: setting an array element with a sequence.
Thank you in advance.
python feature-selection survival-analysis cox-regression lifelines
asked Nov 25 '18 at 12:17
Antonio DichevAntonio Dichev
213
i am using this implemented model in Python for the purpose of survival analysis:
from lifelines import CoxPHFitter
Unfortunately i am not able(i do not know how) to loop over all covariates (features) to run the regression individualy for the purpose of feature selection and save their result. I am trying the script below:
`def fit_and_score_features2(X):
y=X[["Status","duration_yrs"]]
X.drop(["duration_yrs", "Status"], axis=1, inplace=True)
n_features = X.shape[1]
scores = np.empty(n_features)
m = CoxPHFitter()
for j in range(n_features):
Xj = X.values[:, j:j+1]
Xj=pd.merge(X, y, how='right', left_index=True, right_index=True)
m.fit(Xj, duration_col="duration_yrs", event_col="Status", show_progress=True)
scores[j] = m._score_
return scores`
Unfortunately it return me this error:
ValueError Traceback (most recent call
last) in ()
1 #Trying the function above
----> 2 scores = fit_and_score_features2(sample)
3 pd.Series(scores, index=features.columns).sort_values(ascending=False)
in fit_and_score_features2(X)
15 Xj=pd.merge(X, y, how='right', left_index=True, right_index=True)
16 m.fit(Xj, duration_col="duration_yrs", event_col="Status", show_progress=True)
---> 17 scores[j] = m.score
18 return scores
ValueError: setting an array element with a sequence.
Thank you in advance.
python feature-selection survival-analysis cox-regression lifelines
python feature-selection survival-analysis cox-regression lifelines
asked Nov 25 '18 at 12:17
Antonio DichevAntonio Dichev
213
asked Nov 25 '18 at 12:17
Antonio DichevAntonio Dichev
213
asked Nov 25 '18 at 12:17
Antonio DichevAntonio Dichev
213
asked Nov 25 '18 at 12:17
Antonio DichevAntonio Dichev
213
asked Nov 25 '18 at 12:17
Antonio DichevAntonio Dichev
213
213
Why are you using_score_- that's a hidden variable, and it does not represent any kind of accuracy performance?score_however is a measure of accuracy.
– Cam.Davidson.Pilon
Nov 25 '18 at 15:58
Oh, yes you are right, but it still does not work properly. The algorithm doesn't save individual values for each variable. Return of function: X1 0.523545 X2 0.523545 X3 0.523545 X4 0.52354
– Antonio Dichev
Nov 25 '18 at 16:16
I think i was able to debug it properly
– Antonio Dichev
Nov 25 '18 at 16:26
add a comment |
Why are you using_score_- that's a hidden variable, and it does not represent any kind of accuracy performance?score_however is a measure of accuracy.
– Cam.Davidson.Pilon
Nov 25 '18 at 15:58
Oh, yes you are right, but it still does not work properly. The algorithm doesn't save individual values for each variable. Return of function: X1 0.523545 X2 0.523545 X3 0.523545 X4 0.52354
– Antonio Dichev
Nov 25 '18 at 16:16
I think i was able to debug it properly
– Antonio Dichev
Nov 25 '18 at 16:26
Why are you using
_score_ - that's a hidden variable, and it does not represent any kind of accuracy performance? score_ however is a measure of accuracy.– Cam.Davidson.Pilon
Nov 25 '18 at 15:58
Why are you using
_score_ - that's a hidden variable, and it does not represent any kind of accuracy performance? score_ however is a measure of accuracy.– Cam.Davidson.Pilon
Nov 25 '18 at 15:58
Oh, yes you are right, but it still does not work properly. The algorithm doesn't save individual values for each variable. Return of function: X1 0.523545 X2 0.523545 X3 0.523545 X4 0.52354
– Antonio Dichev
Nov 25 '18 at 16:16
Oh, yes you are right, but it still does not work properly. The algorithm doesn't save individual values for each variable. Return of function: X1 0.523545 X2 0.523545 X3 0.523545 X4 0.52354
– Antonio Dichev
Nov 25 '18 at 16:16
I think i was able to debug it properly
– Antonio Dichev
Nov 25 '18 at 16:26
I think i was able to debug it properly
– Antonio Dichev
Nov 25 '18 at 16:26
add a comment |
1 Answer
1
active
oldest
votes
I think that i was able to debug with your help (@Cam.Davidson.Pilon). Thanks a lot. It is the proper script in my opinion:
`def fit_and_score_features2(X):
y=X[["Status","duration_yrs"]]
X.drop(["duration_yrs", "Status"], axis=1, inplace=True)
n_features = X.shape[1]
scores = np.empty(n_features)
m = CoxPHFitter()
for j in range(n_features):
Xj = X.iloc[:, j:j+1]
Xj=pd.merge(Xj, y, how='right', left_index=True, right_index=True)
m.fit(Xj, duration_col="duration_yrs", event_col="Status", show_progress=True)
scores[j] = m.score_
return scores`
answered Nov 25 '18 at 16:29
Antonio DichevAntonio Dichev
213
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53467348%2ffeature-covariates-selection-in-coxphfitter-lifelines-survival-analysis%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
I think that i was able to debug with your help (@Cam.Davidson.Pilon). Thanks a lot. It is the proper script in my opinion:
`def fit_and_score_features2(X):
y=X[["Status","duration_yrs"]]
X.drop(["duration_yrs", "Status"], axis=1, inplace=True)
n_features = X.shape[1]
scores = np.empty(n_features)
m = CoxPHFitter()
for j in range(n_features):
Xj = X.iloc[:, j:j+1]
Xj=pd.merge(Xj, y, how='right', left_index=True, right_index=True)
m.fit(Xj, duration_col="duration_yrs", event_col="Status", show_progress=True)
scores[j] = m.score_
return scores`
answered Nov 25 '18 at 16:29
Antonio DichevAntonio Dichev
213
add a comment |
I think that i was able to debug with your help (@Cam.Davidson.Pilon). Thanks a lot. It is the proper script in my opinion:
`def fit_and_score_features2(X):
y=X[["Status","duration_yrs"]]
X.drop(["duration_yrs", "Status"], axis=1, inplace=True)
n_features = X.shape[1]
scores = np.empty(n_features)
m = CoxPHFitter()
for j in range(n_features):
Xj = X.iloc[:, j:j+1]
Xj=pd.merge(Xj, y, how='right', left_index=True, right_index=True)
m.fit(Xj, duration_col="duration_yrs", event_col="Status", show_progress=True)
scores[j] = m.score_
return scores`
answered Nov 25 '18 at 16:29
Antonio DichevAntonio Dichev
213
add a comment |
I think that i was able to debug with your help (@Cam.Davidson.Pilon). Thanks a lot. It is the proper script in my opinion:
`def fit_and_score_features2(X):
y=X[["Status","duration_yrs"]]
X.drop(["duration_yrs", "Status"], axis=1, inplace=True)
n_features = X.shape[1]
scores = np.empty(n_features)
m = CoxPHFitter()
for j in range(n_features):
Xj = X.iloc[:, j:j+1]
Xj=pd.merge(Xj, y, how='right', left_index=True, right_index=True)
m.fit(Xj, duration_col="duration_yrs", event_col="Status", show_progress=True)
scores[j] = m.score_
return scores`
answered Nov 25 '18 at 16:29
Antonio DichevAntonio Dichev
213
I think that i was able to debug with your help (@Cam.Davidson.Pilon). Thanks a lot. It is the proper script in my opinion:
`def fit_and_score_features2(X):
y=X[["Status","duration_yrs"]]
X.drop(["duration_yrs", "Status"], axis=1, inplace=True)
n_features = X.shape[1]
scores = np.empty(n_features)
m = CoxPHFitter()
for j in range(n_features):
Xj = X.iloc[:, j:j+1]
Xj=pd.merge(Xj, y, how='right', left_index=True, right_index=True)
m.fit(Xj, duration_col="duration_yrs", event_col="Status", show_progress=True)
scores[j] = m.score_
return scores`
answered Nov 25 '18 at 16:29
Antonio DichevAntonio Dichev
213
edited Nov 25 '18 at 16:52
answered Nov 25 '18 at 16:29
Antonio DichevAntonio Dichev
213
answered Nov 25 '18 at 16:29
Antonio DichevAntonio Dichev
213
answered Nov 25 '18 at 16:29
Antonio DichevAntonio Dichev
213
213
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53467348%2ffeature-covariates-selection-in-coxphfitter-lifelines-survival-analysis%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Why are you using
_score_- that's a hidden variable, and it does not represent any kind of accuracy performance?score_however is a measure of accuracy.– Cam.Davidson.Pilon
Nov 25 '18 at 15:58
Oh, yes you are right, but it still does not work properly. The algorithm doesn't save individual values for each variable. Return of function: X1 0.523545 X2 0.523545 X3 0.523545 X4 0.52354
– Antonio Dichev
Nov 25 '18 at 16:16
I think i was able to debug it properly
– Antonio Dichev
Nov 25 '18 at 16:26