list comprehension and transforming data
I've been trying to convert some of my functions where I use a for loop by using list comprehension. Here is my first version of the function,
def adstocked_advertising(data, adstock_rate):
'''
Transforming data with applying Adstock transformations
data - > The dataframe that is being used to create Adstock variables
adstock_rate -> The rate at of the adstock
ex. data['Channel_Adstock'] = adstocked_advertising(data['Channel'], 0.5)
'''
adstocked_advertising =
for i in range(len(data)):
if i == 0:
adstocked_advertising.append(data[i])
else:
adstocked_advertising.append(data[i] + adstock_rate * adstocked_advertising[i-1])
return adstocked_advertising
I want to convert it to this,
def adstocked_advertising_list(data, adstock_rate):
adstocked_advertising = [data[i] if i == 0 else data[i] + adstock_rate * data[i-1] for i in range(len(data))]
return adstocked_advertising
However, when viewing the df after running both functions I get two different values.
data['TV_adstock'] = adstocked_advertising_list(data['TV'], 0.5)
data['TV_adstock_2'] = adstocked_advertising(data['TV'], 0.5)
here is output,
data.head()
data.tail()
I am not too sure why the first two rows are the same and then from there the numbers are all different. I am new to list comprehension so I may be missing something here.
python list-comprehension
add a comment |
I've been trying to convert some of my functions where I use a for loop by using list comprehension. Here is my first version of the function,
def adstocked_advertising(data, adstock_rate):
'''
Transforming data with applying Adstock transformations
data - > The dataframe that is being used to create Adstock variables
adstock_rate -> The rate at of the adstock
ex. data['Channel_Adstock'] = adstocked_advertising(data['Channel'], 0.5)
'''
adstocked_advertising =
for i in range(len(data)):
if i == 0:
adstocked_advertising.append(data[i])
else:
adstocked_advertising.append(data[i] + adstock_rate * adstocked_advertising[i-1])
return adstocked_advertising
I want to convert it to this,
def adstocked_advertising_list(data, adstock_rate):
adstocked_advertising = [data[i] if i == 0 else data[i] + adstock_rate * data[i-1] for i in range(len(data))]
return adstocked_advertising
However, when viewing the df after running both functions I get two different values.
data['TV_adstock'] = adstocked_advertising_list(data['TV'], 0.5)
data['TV_adstock_2'] = adstocked_advertising(data['TV'], 0.5)
here is output,
data.head()
data.tail()
I am not too sure why the first two rows are the same and then from there the numbers are all different. I am new to list comprehension so I may be missing something here.
python list-comprehension
1
Is there a specific reason you want to convert it to a list comprehension? Sometimes the loop is more legible. I am not aware of significant computational advantages to list comprehension, see here: stackoverflow.com/a/22108640/8793975
– enumaris
Nov 20 at 19:24
1
In the function versionadstocked_advertising_list()
, you referencei
but i is not defined as a parameter, it's just dangling, presumably a dangling non-zero final value from the for-loop version, hence in your function thedata[i] if i == 0
clause is never used. So just fix that and it should work.
– smci
Nov 20 at 19:28
@enumaris , the main reason for this is to learn how to use list comprehension.
– Timothy Mcwilliams
Nov 20 at 19:29
add a comment |
I've been trying to convert some of my functions where I use a for loop by using list comprehension. Here is my first version of the function,
def adstocked_advertising(data, adstock_rate):
'''
Transforming data with applying Adstock transformations
data - > The dataframe that is being used to create Adstock variables
adstock_rate -> The rate at of the adstock
ex. data['Channel_Adstock'] = adstocked_advertising(data['Channel'], 0.5)
'''
adstocked_advertising =
for i in range(len(data)):
if i == 0:
adstocked_advertising.append(data[i])
else:
adstocked_advertising.append(data[i] + adstock_rate * adstocked_advertising[i-1])
return adstocked_advertising
I want to convert it to this,
def adstocked_advertising_list(data, adstock_rate):
adstocked_advertising = [data[i] if i == 0 else data[i] + adstock_rate * data[i-1] for i in range(len(data))]
return adstocked_advertising
However, when viewing the df after running both functions I get two different values.
data['TV_adstock'] = adstocked_advertising_list(data['TV'], 0.5)
data['TV_adstock_2'] = adstocked_advertising(data['TV'], 0.5)
here is output,
data.head()
data.tail()
I am not too sure why the first two rows are the same and then from there the numbers are all different. I am new to list comprehension so I may be missing something here.
python list-comprehension
I've been trying to convert some of my functions where I use a for loop by using list comprehension. Here is my first version of the function,
def adstocked_advertising(data, adstock_rate):
'''
Transforming data with applying Adstock transformations
data - > The dataframe that is being used to create Adstock variables
adstock_rate -> The rate at of the adstock
ex. data['Channel_Adstock'] = adstocked_advertising(data['Channel'], 0.5)
'''
adstocked_advertising =
for i in range(len(data)):
if i == 0:
adstocked_advertising.append(data[i])
else:
adstocked_advertising.append(data[i] + adstock_rate * adstocked_advertising[i-1])
return adstocked_advertising
I want to convert it to this,
def adstocked_advertising_list(data, adstock_rate):
adstocked_advertising = [data[i] if i == 0 else data[i] + adstock_rate * data[i-1] for i in range(len(data))]
return adstocked_advertising
However, when viewing the df after running both functions I get two different values.
data['TV_adstock'] = adstocked_advertising_list(data['TV'], 0.5)
data['TV_adstock_2'] = adstocked_advertising(data['TV'], 0.5)
here is output,
data.head()
data.tail()
I am not too sure why the first two rows are the same and then from there the numbers are all different. I am new to list comprehension so I may be missing something here.
python list-comprehension
python list-comprehension
asked Nov 20 at 19:16
Timothy Mcwilliams
386
386
1
Is there a specific reason you want to convert it to a list comprehension? Sometimes the loop is more legible. I am not aware of significant computational advantages to list comprehension, see here: stackoverflow.com/a/22108640/8793975
– enumaris
Nov 20 at 19:24
1
In the function versionadstocked_advertising_list()
, you referencei
but i is not defined as a parameter, it's just dangling, presumably a dangling non-zero final value from the for-loop version, hence in your function thedata[i] if i == 0
clause is never used. So just fix that and it should work.
– smci
Nov 20 at 19:28
@enumaris , the main reason for this is to learn how to use list comprehension.
– Timothy Mcwilliams
Nov 20 at 19:29
add a comment |
1
Is there a specific reason you want to convert it to a list comprehension? Sometimes the loop is more legible. I am not aware of significant computational advantages to list comprehension, see here: stackoverflow.com/a/22108640/8793975
– enumaris
Nov 20 at 19:24
1
In the function versionadstocked_advertising_list()
, you referencei
but i is not defined as a parameter, it's just dangling, presumably a dangling non-zero final value from the for-loop version, hence in your function thedata[i] if i == 0
clause is never used. So just fix that and it should work.
– smci
Nov 20 at 19:28
@enumaris , the main reason for this is to learn how to use list comprehension.
– Timothy Mcwilliams
Nov 20 at 19:29
1
1
Is there a specific reason you want to convert it to a list comprehension? Sometimes the loop is more legible. I am not aware of significant computational advantages to list comprehension, see here: stackoverflow.com/a/22108640/8793975
– enumaris
Nov 20 at 19:24
Is there a specific reason you want to convert it to a list comprehension? Sometimes the loop is more legible. I am not aware of significant computational advantages to list comprehension, see here: stackoverflow.com/a/22108640/8793975
– enumaris
Nov 20 at 19:24
1
1
In the function version
adstocked_advertising_list()
, you reference i
but i is not defined as a parameter, it's just dangling, presumably a dangling non-zero final value from the for-loop version, hence in your function the data[i] if i == 0
clause is never used. So just fix that and it should work.– smci
Nov 20 at 19:28
In the function version
adstocked_advertising_list()
, you reference i
but i is not defined as a parameter, it's just dangling, presumably a dangling non-zero final value from the for-loop version, hence in your function the data[i] if i == 0
clause is never used. So just fix that and it should work.– smci
Nov 20 at 19:28
@enumaris , the main reason for this is to learn how to use list comprehension.
– Timothy Mcwilliams
Nov 20 at 19:29
@enumaris , the main reason for this is to learn how to use list comprehension.
– Timothy Mcwilliams
Nov 20 at 19:29
add a comment |
2 Answers
2
active
oldest
votes
You need to refer to the previously generated element in the list, and list comprehensions are not well suited to this type of problem. They work well for operations that only need to look at a single element at once.
This question goes into more detail.
In your initial example, you use adstock_rate * adstocked_advertising[i-1]
. The list comprehension version uses adstock_rate * data[i-1]
, which is why you are getting different results.
A standard for loop works just fine for your use case. You could switch to using enumerate, as for i in range(len(data))
is discouraged.
if data:
res = [data[0]]
for index, item in enumerate(data[1:]):
results.append(item + rate * data[index-1])
1
Thanks for the suggestion. I guess there are specific use cases where list compression is used over a standard for loop and vise versa.
– Timothy Mcwilliams
Nov 20 at 19:36
2
if index == 0
will only be true once, at the beginning of the loop. Append the first element, and then iterate unconditionally.
– ggorlen
Nov 20 at 19:38
Your translation of the algorithm is now incorrect because the first element will be appended twice.
– ggorlen
Nov 20 at 20:16
add a comment |
You've changed your logic in the list comp version. Originally, your else
formula looked like:
data[i] + adstock_rate * adstocked_advertising[i-1]
But the list comprehension version looks like:
data[i] + adstock_rate * data[i-1]
The first version accesses the i-1
th element of the result list, while the second version accesses the i-1
th element of the input list.
index == 0
is only true once at the beginning of the list. Why not eliminate the conditional:
def adstocked_advertising(data, adstock_rate):
if data:
res = [data[0]]
for i in range(1, len(data)):
res.append(data[i] + adstock_rate * res[i-1])
return res
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53400014%2flist-comprehension-and-transforming-data%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
You need to refer to the previously generated element in the list, and list comprehensions are not well suited to this type of problem. They work well for operations that only need to look at a single element at once.
This question goes into more detail.
In your initial example, you use adstock_rate * adstocked_advertising[i-1]
. The list comprehension version uses adstock_rate * data[i-1]
, which is why you are getting different results.
A standard for loop works just fine for your use case. You could switch to using enumerate, as for i in range(len(data))
is discouraged.
if data:
res = [data[0]]
for index, item in enumerate(data[1:]):
results.append(item + rate * data[index-1])
1
Thanks for the suggestion. I guess there are specific use cases where list compression is used over a standard for loop and vise versa.
– Timothy Mcwilliams
Nov 20 at 19:36
2
if index == 0
will only be true once, at the beginning of the loop. Append the first element, and then iterate unconditionally.
– ggorlen
Nov 20 at 19:38
Your translation of the algorithm is now incorrect because the first element will be appended twice.
– ggorlen
Nov 20 at 20:16
add a comment |
You need to refer to the previously generated element in the list, and list comprehensions are not well suited to this type of problem. They work well for operations that only need to look at a single element at once.
This question goes into more detail.
In your initial example, you use adstock_rate * adstocked_advertising[i-1]
. The list comprehension version uses adstock_rate * data[i-1]
, which is why you are getting different results.
A standard for loop works just fine for your use case. You could switch to using enumerate, as for i in range(len(data))
is discouraged.
if data:
res = [data[0]]
for index, item in enumerate(data[1:]):
results.append(item + rate * data[index-1])
1
Thanks for the suggestion. I guess there are specific use cases where list compression is used over a standard for loop and vise versa.
– Timothy Mcwilliams
Nov 20 at 19:36
2
if index == 0
will only be true once, at the beginning of the loop. Append the first element, and then iterate unconditionally.
– ggorlen
Nov 20 at 19:38
Your translation of the algorithm is now incorrect because the first element will be appended twice.
– ggorlen
Nov 20 at 20:16
add a comment |
You need to refer to the previously generated element in the list, and list comprehensions are not well suited to this type of problem. They work well for operations that only need to look at a single element at once.
This question goes into more detail.
In your initial example, you use adstock_rate * adstocked_advertising[i-1]
. The list comprehension version uses adstock_rate * data[i-1]
, which is why you are getting different results.
A standard for loop works just fine for your use case. You could switch to using enumerate, as for i in range(len(data))
is discouraged.
if data:
res = [data[0]]
for index, item in enumerate(data[1:]):
results.append(item + rate * data[index-1])
You need to refer to the previously generated element in the list, and list comprehensions are not well suited to this type of problem. They work well for operations that only need to look at a single element at once.
This question goes into more detail.
In your initial example, you use adstock_rate * adstocked_advertising[i-1]
. The list comprehension version uses adstock_rate * data[i-1]
, which is why you are getting different results.
A standard for loop works just fine for your use case. You could switch to using enumerate, as for i in range(len(data))
is discouraged.
if data:
res = [data[0]]
for index, item in enumerate(data[1:]):
results.append(item + rate * data[index-1])
edited Nov 20 at 20:42
answered Nov 20 at 19:30
Wieschie
169110
169110
1
Thanks for the suggestion. I guess there are specific use cases where list compression is used over a standard for loop and vise versa.
– Timothy Mcwilliams
Nov 20 at 19:36
2
if index == 0
will only be true once, at the beginning of the loop. Append the first element, and then iterate unconditionally.
– ggorlen
Nov 20 at 19:38
Your translation of the algorithm is now incorrect because the first element will be appended twice.
– ggorlen
Nov 20 at 20:16
add a comment |
1
Thanks for the suggestion. I guess there are specific use cases where list compression is used over a standard for loop and vise versa.
– Timothy Mcwilliams
Nov 20 at 19:36
2
if index == 0
will only be true once, at the beginning of the loop. Append the first element, and then iterate unconditionally.
– ggorlen
Nov 20 at 19:38
Your translation of the algorithm is now incorrect because the first element will be appended twice.
– ggorlen
Nov 20 at 20:16
1
1
Thanks for the suggestion. I guess there are specific use cases where list compression is used over a standard for loop and vise versa.
– Timothy Mcwilliams
Nov 20 at 19:36
Thanks for the suggestion. I guess there are specific use cases where list compression is used over a standard for loop and vise versa.
– Timothy Mcwilliams
Nov 20 at 19:36
2
2
if index == 0
will only be true once, at the beginning of the loop. Append the first element, and then iterate unconditionally.– ggorlen
Nov 20 at 19:38
if index == 0
will only be true once, at the beginning of the loop. Append the first element, and then iterate unconditionally.– ggorlen
Nov 20 at 19:38
Your translation of the algorithm is now incorrect because the first element will be appended twice.
– ggorlen
Nov 20 at 20:16
Your translation of the algorithm is now incorrect because the first element will be appended twice.
– ggorlen
Nov 20 at 20:16
add a comment |
You've changed your logic in the list comp version. Originally, your else
formula looked like:
data[i] + adstock_rate * adstocked_advertising[i-1]
But the list comprehension version looks like:
data[i] + adstock_rate * data[i-1]
The first version accesses the i-1
th element of the result list, while the second version accesses the i-1
th element of the input list.
index == 0
is only true once at the beginning of the list. Why not eliminate the conditional:
def adstocked_advertising(data, adstock_rate):
if data:
res = [data[0]]
for i in range(1, len(data)):
res.append(data[i] + adstock_rate * res[i-1])
return res
add a comment |
You've changed your logic in the list comp version. Originally, your else
formula looked like:
data[i] + adstock_rate * adstocked_advertising[i-1]
But the list comprehension version looks like:
data[i] + adstock_rate * data[i-1]
The first version accesses the i-1
th element of the result list, while the second version accesses the i-1
th element of the input list.
index == 0
is only true once at the beginning of the list. Why not eliminate the conditional:
def adstocked_advertising(data, adstock_rate):
if data:
res = [data[0]]
for i in range(1, len(data)):
res.append(data[i] + adstock_rate * res[i-1])
return res
add a comment |
You've changed your logic in the list comp version. Originally, your else
formula looked like:
data[i] + adstock_rate * adstocked_advertising[i-1]
But the list comprehension version looks like:
data[i] + adstock_rate * data[i-1]
The first version accesses the i-1
th element of the result list, while the second version accesses the i-1
th element of the input list.
index == 0
is only true once at the beginning of the list. Why not eliminate the conditional:
def adstocked_advertising(data, adstock_rate):
if data:
res = [data[0]]
for i in range(1, len(data)):
res.append(data[i] + adstock_rate * res[i-1])
return res
You've changed your logic in the list comp version. Originally, your else
formula looked like:
data[i] + adstock_rate * adstocked_advertising[i-1]
But the list comprehension version looks like:
data[i] + adstock_rate * data[i-1]
The first version accesses the i-1
th element of the result list, while the second version accesses the i-1
th element of the input list.
index == 0
is only true once at the beginning of the list. Why not eliminate the conditional:
def adstocked_advertising(data, adstock_rate):
if data:
res = [data[0]]
for i in range(1, len(data)):
res.append(data[i] + adstock_rate * res[i-1])
return res
edited Nov 20 at 19:42
answered Nov 20 at 19:28
ggorlen
6,3413825
6,3413825
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53400014%2flist-comprehension-and-transforming-data%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
Is there a specific reason you want to convert it to a list comprehension? Sometimes the loop is more legible. I am not aware of significant computational advantages to list comprehension, see here: stackoverflow.com/a/22108640/8793975
– enumaris
Nov 20 at 19:24
1
In the function version
adstocked_advertising_list()
, you referencei
but i is not defined as a parameter, it's just dangling, presumably a dangling non-zero final value from the for-loop version, hence in your function thedata[i] if i == 0
clause is never used. So just fix that and it should work.– smci
Nov 20 at 19:28
@enumaris , the main reason for this is to learn how to use list comprehension.
– Timothy Mcwilliams
Nov 20 at 19:29