How to replace Inf values with NaN in array in Julia 1.0?
I have seen online in a few places the solution
a = [1 2 3; 4 5 Inf]
a[isinf(a)] = NaN
But this gives me an error on Julia 1.0.1:
ERROR: MethodError: no method matching isinf(::Array{Float64,2})
Closest candidates are:
isinf(::BigFloat) at mpfr.jl:851
isinf(::Missing) at missing.jl:79
isinf(::ForwardDiff.Dual) at <path on my local machine>
What gives?
julia
add a comment |
I have seen online in a few places the solution
a = [1 2 3; 4 5 Inf]
a[isinf(a)] = NaN
But this gives me an error on Julia 1.0.1:
ERROR: MethodError: no method matching isinf(::Array{Float64,2})
Closest candidates are:
isinf(::BigFloat) at mpfr.jl:851
isinf(::Missing) at missing.jl:79
isinf(::ForwardDiff.Dual) at <path on my local machine>
What gives?
julia
add a comment |
I have seen online in a few places the solution
a = [1 2 3; 4 5 Inf]
a[isinf(a)] = NaN
But this gives me an error on Julia 1.0.1:
ERROR: MethodError: no method matching isinf(::Array{Float64,2})
Closest candidates are:
isinf(::BigFloat) at mpfr.jl:851
isinf(::Missing) at missing.jl:79
isinf(::ForwardDiff.Dual) at <path on my local machine>
What gives?
julia
I have seen online in a few places the solution
a = [1 2 3; 4 5 Inf]
a[isinf(a)] = NaN
But this gives me an error on Julia 1.0.1:
ERROR: MethodError: no method matching isinf(::Array{Float64,2})
Closest candidates are:
isinf(::BigFloat) at mpfr.jl:851
isinf(::Missing) at missing.jl:79
isinf(::ForwardDiff.Dual) at <path on my local machine>
What gives?
julia
julia
asked Nov 24 '18 at 23:49
nfernandnfernand
1201110
1201110
add a comment |
add a comment |
3 Answers
3
active
oldest
votes
As an additional comment. A standard function to perform this action is replace!
. You can use it like this:
julia> a = [1 2 3; 4 5 Inf]
2×3 Array{Float64,2}:
1.0 2.0 3.0
4.0 5.0 Inf
julia> replace!(a, Inf=>NaN)
2×3 Array{Float64,2}:
1.0 2.0 3.0
4.0 5.0 NaN
It will perform better than broadcasting for large arrays.
If you really need speed you can write a simple function like this:
function inf2nan(x)
for i in eachindex(x)
@inbounds x[i] = ifelse(isinf(x[i]), NaN, x[i])
end
end
Now let us simply compare the performance of the three options:
julia> function bench()
x = fill(Inf, 10^8)
@time x[isinf.(x)] .= NaN
x = fill(Inf, 10^8)
@time replace!(x, Inf=>NaN)
x = fill(Inf, 10^8)
@time inf2nan(x)
end
bench (generic function with 1 method)
julia> bench()
0.980434 seconds (9 allocations: 774.865 MiB, 0.16% gc time)
0.183578 seconds
0.109929 seconds
julia> bench()
0.971408 seconds (9 allocations: 774.865 MiB, 0.03% gc time)
0.184163 seconds
0.102161 seconds
This is a very useful answer. I typed my answer without thinking about the fact thatisinf.(x)
would allocate.
– Colin T Bowers
Nov 25 '18 at 8:40
2
Yes, in Julia - opposite do e.g. R - if you need performance you usually will want to avoid allocation of a binary vector to perform subsetting. In this particular case the loop is faster thanreplace!
in this case because we can avoid branching in the loop usingifelse
and since the operation we perform is simple and@inbounds
the compiler can significantly optimize its execution time.
– Bogumił Kamiński
Nov 25 '18 at 8:50
Very interesting, thanks for explaining. I'll direct readers to your answer, but leave my answer as it is, since it addresses the part of the question that wanted to know whyisinf
doesn't work on arrays anymore.
– Colin T Bowers
Nov 25 '18 at 11:02
Your answer is the simplest approach in 90% of cases it is good enough and all you need to know, so +1 :).
– Bogumił Kamiński
Nov 25 '18 at 12:58
add a comment |
EDIT: For the most performant approaches to this problem see the excellent answer of @BogumilKaminski. This answer addresses the more general question of why isinf
and related functions do not work on arrays anymore.
You are running into the more general issue that lots of functions that worked on arrays pre-v1.0 no longer work on arrays in v1.0 because you are supposed to be using broadcasting. The correct solution for v1.0 is:
a[isinf.(a)] .= NaN
I'm actually broadcasting in two places here. Firstly, we broadcast isinf
over the array a
, but we are also broadcasting the scalar NaN
on the RHS to all indexed locations in the array on the LHS via .=
. In general, the dot broadcasting notation is incredibly flexible and performant, and one of my favorite features of the latest iteration of Julia.
add a comment |
You are passing your entire array to isinf, it doesn't work on arrays, it works on numbers. Try this:
[isinf(i) ? NaN : i for i in a]
2
This actually creates a new array rather than updating the current array. See my answer for more detail.
– Colin T Bowers
Nov 25 '18 at 0:39
Yeah fair enough, your answer is better
– chasmani
Nov 26 '18 at 18:02
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53463436%2fhow-to-replace-inf-values-with-nan-in-array-in-julia-1-0%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
As an additional comment. A standard function to perform this action is replace!
. You can use it like this:
julia> a = [1 2 3; 4 5 Inf]
2×3 Array{Float64,2}:
1.0 2.0 3.0
4.0 5.0 Inf
julia> replace!(a, Inf=>NaN)
2×3 Array{Float64,2}:
1.0 2.0 3.0
4.0 5.0 NaN
It will perform better than broadcasting for large arrays.
If you really need speed you can write a simple function like this:
function inf2nan(x)
for i in eachindex(x)
@inbounds x[i] = ifelse(isinf(x[i]), NaN, x[i])
end
end
Now let us simply compare the performance of the three options:
julia> function bench()
x = fill(Inf, 10^8)
@time x[isinf.(x)] .= NaN
x = fill(Inf, 10^8)
@time replace!(x, Inf=>NaN)
x = fill(Inf, 10^8)
@time inf2nan(x)
end
bench (generic function with 1 method)
julia> bench()
0.980434 seconds (9 allocations: 774.865 MiB, 0.16% gc time)
0.183578 seconds
0.109929 seconds
julia> bench()
0.971408 seconds (9 allocations: 774.865 MiB, 0.03% gc time)
0.184163 seconds
0.102161 seconds
This is a very useful answer. I typed my answer without thinking about the fact thatisinf.(x)
would allocate.
– Colin T Bowers
Nov 25 '18 at 8:40
2
Yes, in Julia - opposite do e.g. R - if you need performance you usually will want to avoid allocation of a binary vector to perform subsetting. In this particular case the loop is faster thanreplace!
in this case because we can avoid branching in the loop usingifelse
and since the operation we perform is simple and@inbounds
the compiler can significantly optimize its execution time.
– Bogumił Kamiński
Nov 25 '18 at 8:50
Very interesting, thanks for explaining. I'll direct readers to your answer, but leave my answer as it is, since it addresses the part of the question that wanted to know whyisinf
doesn't work on arrays anymore.
– Colin T Bowers
Nov 25 '18 at 11:02
Your answer is the simplest approach in 90% of cases it is good enough and all you need to know, so +1 :).
– Bogumił Kamiński
Nov 25 '18 at 12:58
add a comment |
As an additional comment. A standard function to perform this action is replace!
. You can use it like this:
julia> a = [1 2 3; 4 5 Inf]
2×3 Array{Float64,2}:
1.0 2.0 3.0
4.0 5.0 Inf
julia> replace!(a, Inf=>NaN)
2×3 Array{Float64,2}:
1.0 2.0 3.0
4.0 5.0 NaN
It will perform better than broadcasting for large arrays.
If you really need speed you can write a simple function like this:
function inf2nan(x)
for i in eachindex(x)
@inbounds x[i] = ifelse(isinf(x[i]), NaN, x[i])
end
end
Now let us simply compare the performance of the three options:
julia> function bench()
x = fill(Inf, 10^8)
@time x[isinf.(x)] .= NaN
x = fill(Inf, 10^8)
@time replace!(x, Inf=>NaN)
x = fill(Inf, 10^8)
@time inf2nan(x)
end
bench (generic function with 1 method)
julia> bench()
0.980434 seconds (9 allocations: 774.865 MiB, 0.16% gc time)
0.183578 seconds
0.109929 seconds
julia> bench()
0.971408 seconds (9 allocations: 774.865 MiB, 0.03% gc time)
0.184163 seconds
0.102161 seconds
This is a very useful answer. I typed my answer without thinking about the fact thatisinf.(x)
would allocate.
– Colin T Bowers
Nov 25 '18 at 8:40
2
Yes, in Julia - opposite do e.g. R - if you need performance you usually will want to avoid allocation of a binary vector to perform subsetting. In this particular case the loop is faster thanreplace!
in this case because we can avoid branching in the loop usingifelse
and since the operation we perform is simple and@inbounds
the compiler can significantly optimize its execution time.
– Bogumił Kamiński
Nov 25 '18 at 8:50
Very interesting, thanks for explaining. I'll direct readers to your answer, but leave my answer as it is, since it addresses the part of the question that wanted to know whyisinf
doesn't work on arrays anymore.
– Colin T Bowers
Nov 25 '18 at 11:02
Your answer is the simplest approach in 90% of cases it is good enough and all you need to know, so +1 :).
– Bogumił Kamiński
Nov 25 '18 at 12:58
add a comment |
As an additional comment. A standard function to perform this action is replace!
. You can use it like this:
julia> a = [1 2 3; 4 5 Inf]
2×3 Array{Float64,2}:
1.0 2.0 3.0
4.0 5.0 Inf
julia> replace!(a, Inf=>NaN)
2×3 Array{Float64,2}:
1.0 2.0 3.0
4.0 5.0 NaN
It will perform better than broadcasting for large arrays.
If you really need speed you can write a simple function like this:
function inf2nan(x)
for i in eachindex(x)
@inbounds x[i] = ifelse(isinf(x[i]), NaN, x[i])
end
end
Now let us simply compare the performance of the three options:
julia> function bench()
x = fill(Inf, 10^8)
@time x[isinf.(x)] .= NaN
x = fill(Inf, 10^8)
@time replace!(x, Inf=>NaN)
x = fill(Inf, 10^8)
@time inf2nan(x)
end
bench (generic function with 1 method)
julia> bench()
0.980434 seconds (9 allocations: 774.865 MiB, 0.16% gc time)
0.183578 seconds
0.109929 seconds
julia> bench()
0.971408 seconds (9 allocations: 774.865 MiB, 0.03% gc time)
0.184163 seconds
0.102161 seconds
As an additional comment. A standard function to perform this action is replace!
. You can use it like this:
julia> a = [1 2 3; 4 5 Inf]
2×3 Array{Float64,2}:
1.0 2.0 3.0
4.0 5.0 Inf
julia> replace!(a, Inf=>NaN)
2×3 Array{Float64,2}:
1.0 2.0 3.0
4.0 5.0 NaN
It will perform better than broadcasting for large arrays.
If you really need speed you can write a simple function like this:
function inf2nan(x)
for i in eachindex(x)
@inbounds x[i] = ifelse(isinf(x[i]), NaN, x[i])
end
end
Now let us simply compare the performance of the three options:
julia> function bench()
x = fill(Inf, 10^8)
@time x[isinf.(x)] .= NaN
x = fill(Inf, 10^8)
@time replace!(x, Inf=>NaN)
x = fill(Inf, 10^8)
@time inf2nan(x)
end
bench (generic function with 1 method)
julia> bench()
0.980434 seconds (9 allocations: 774.865 MiB, 0.16% gc time)
0.183578 seconds
0.109929 seconds
julia> bench()
0.971408 seconds (9 allocations: 774.865 MiB, 0.03% gc time)
0.184163 seconds
0.102161 seconds
answered Nov 25 '18 at 7:57
Bogumił KamińskiBogumił Kamiński
13.8k11220
13.8k11220
This is a very useful answer. I typed my answer without thinking about the fact thatisinf.(x)
would allocate.
– Colin T Bowers
Nov 25 '18 at 8:40
2
Yes, in Julia - opposite do e.g. R - if you need performance you usually will want to avoid allocation of a binary vector to perform subsetting. In this particular case the loop is faster thanreplace!
in this case because we can avoid branching in the loop usingifelse
and since the operation we perform is simple and@inbounds
the compiler can significantly optimize its execution time.
– Bogumił Kamiński
Nov 25 '18 at 8:50
Very interesting, thanks for explaining. I'll direct readers to your answer, but leave my answer as it is, since it addresses the part of the question that wanted to know whyisinf
doesn't work on arrays anymore.
– Colin T Bowers
Nov 25 '18 at 11:02
Your answer is the simplest approach in 90% of cases it is good enough and all you need to know, so +1 :).
– Bogumił Kamiński
Nov 25 '18 at 12:58
add a comment |
This is a very useful answer. I typed my answer without thinking about the fact thatisinf.(x)
would allocate.
– Colin T Bowers
Nov 25 '18 at 8:40
2
Yes, in Julia - opposite do e.g. R - if you need performance you usually will want to avoid allocation of a binary vector to perform subsetting. In this particular case the loop is faster thanreplace!
in this case because we can avoid branching in the loop usingifelse
and since the operation we perform is simple and@inbounds
the compiler can significantly optimize its execution time.
– Bogumił Kamiński
Nov 25 '18 at 8:50
Very interesting, thanks for explaining. I'll direct readers to your answer, but leave my answer as it is, since it addresses the part of the question that wanted to know whyisinf
doesn't work on arrays anymore.
– Colin T Bowers
Nov 25 '18 at 11:02
Your answer is the simplest approach in 90% of cases it is good enough and all you need to know, so +1 :).
– Bogumił Kamiński
Nov 25 '18 at 12:58
This is a very useful answer. I typed my answer without thinking about the fact that
isinf.(x)
would allocate.– Colin T Bowers
Nov 25 '18 at 8:40
This is a very useful answer. I typed my answer without thinking about the fact that
isinf.(x)
would allocate.– Colin T Bowers
Nov 25 '18 at 8:40
2
2
Yes, in Julia - opposite do e.g. R - if you need performance you usually will want to avoid allocation of a binary vector to perform subsetting. In this particular case the loop is faster than
replace!
in this case because we can avoid branching in the loop using ifelse
and since the operation we perform is simple and @inbounds
the compiler can significantly optimize its execution time.– Bogumił Kamiński
Nov 25 '18 at 8:50
Yes, in Julia - opposite do e.g. R - if you need performance you usually will want to avoid allocation of a binary vector to perform subsetting. In this particular case the loop is faster than
replace!
in this case because we can avoid branching in the loop using ifelse
and since the operation we perform is simple and @inbounds
the compiler can significantly optimize its execution time.– Bogumił Kamiński
Nov 25 '18 at 8:50
Very interesting, thanks for explaining. I'll direct readers to your answer, but leave my answer as it is, since it addresses the part of the question that wanted to know why
isinf
doesn't work on arrays anymore.– Colin T Bowers
Nov 25 '18 at 11:02
Very interesting, thanks for explaining. I'll direct readers to your answer, but leave my answer as it is, since it addresses the part of the question that wanted to know why
isinf
doesn't work on arrays anymore.– Colin T Bowers
Nov 25 '18 at 11:02
Your answer is the simplest approach in 90% of cases it is good enough and all you need to know, so +1 :).
– Bogumił Kamiński
Nov 25 '18 at 12:58
Your answer is the simplest approach in 90% of cases it is good enough and all you need to know, so +1 :).
– Bogumił Kamiński
Nov 25 '18 at 12:58
add a comment |
EDIT: For the most performant approaches to this problem see the excellent answer of @BogumilKaminski. This answer addresses the more general question of why isinf
and related functions do not work on arrays anymore.
You are running into the more general issue that lots of functions that worked on arrays pre-v1.0 no longer work on arrays in v1.0 because you are supposed to be using broadcasting. The correct solution for v1.0 is:
a[isinf.(a)] .= NaN
I'm actually broadcasting in two places here. Firstly, we broadcast isinf
over the array a
, but we are also broadcasting the scalar NaN
on the RHS to all indexed locations in the array on the LHS via .=
. In general, the dot broadcasting notation is incredibly flexible and performant, and one of my favorite features of the latest iteration of Julia.
add a comment |
EDIT: For the most performant approaches to this problem see the excellent answer of @BogumilKaminski. This answer addresses the more general question of why isinf
and related functions do not work on arrays anymore.
You are running into the more general issue that lots of functions that worked on arrays pre-v1.0 no longer work on arrays in v1.0 because you are supposed to be using broadcasting. The correct solution for v1.0 is:
a[isinf.(a)] .= NaN
I'm actually broadcasting in two places here. Firstly, we broadcast isinf
over the array a
, but we are also broadcasting the scalar NaN
on the RHS to all indexed locations in the array on the LHS via .=
. In general, the dot broadcasting notation is incredibly flexible and performant, and one of my favorite features of the latest iteration of Julia.
add a comment |
EDIT: For the most performant approaches to this problem see the excellent answer of @BogumilKaminski. This answer addresses the more general question of why isinf
and related functions do not work on arrays anymore.
You are running into the more general issue that lots of functions that worked on arrays pre-v1.0 no longer work on arrays in v1.0 because you are supposed to be using broadcasting. The correct solution for v1.0 is:
a[isinf.(a)] .= NaN
I'm actually broadcasting in two places here. Firstly, we broadcast isinf
over the array a
, but we are also broadcasting the scalar NaN
on the RHS to all indexed locations in the array on the LHS via .=
. In general, the dot broadcasting notation is incredibly flexible and performant, and one of my favorite features of the latest iteration of Julia.
EDIT: For the most performant approaches to this problem see the excellent answer of @BogumilKaminski. This answer addresses the more general question of why isinf
and related functions do not work on arrays anymore.
You are running into the more general issue that lots of functions that worked on arrays pre-v1.0 no longer work on arrays in v1.0 because you are supposed to be using broadcasting. The correct solution for v1.0 is:
a[isinf.(a)] .= NaN
I'm actually broadcasting in two places here. Firstly, we broadcast isinf
over the array a
, but we are also broadcasting the scalar NaN
on the RHS to all indexed locations in the array on the LHS via .=
. In general, the dot broadcasting notation is incredibly flexible and performant, and one of my favorite features of the latest iteration of Julia.
edited Nov 26 '18 at 21:35
answered Nov 25 '18 at 0:35
Colin T BowersColin T Bowers
10.3k43764
10.3k43764
add a comment |
add a comment |
You are passing your entire array to isinf, it doesn't work on arrays, it works on numbers. Try this:
[isinf(i) ? NaN : i for i in a]
2
This actually creates a new array rather than updating the current array. See my answer for more detail.
– Colin T Bowers
Nov 25 '18 at 0:39
Yeah fair enough, your answer is better
– chasmani
Nov 26 '18 at 18:02
add a comment |
You are passing your entire array to isinf, it doesn't work on arrays, it works on numbers. Try this:
[isinf(i) ? NaN : i for i in a]
2
This actually creates a new array rather than updating the current array. See my answer for more detail.
– Colin T Bowers
Nov 25 '18 at 0:39
Yeah fair enough, your answer is better
– chasmani
Nov 26 '18 at 18:02
add a comment |
You are passing your entire array to isinf, it doesn't work on arrays, it works on numbers. Try this:
[isinf(i) ? NaN : i for i in a]
You are passing your entire array to isinf, it doesn't work on arrays, it works on numbers. Try this:
[isinf(i) ? NaN : i for i in a]
answered Nov 25 '18 at 0:14
chasmanichasmani
1,03211020
1,03211020
2
This actually creates a new array rather than updating the current array. See my answer for more detail.
– Colin T Bowers
Nov 25 '18 at 0:39
Yeah fair enough, your answer is better
– chasmani
Nov 26 '18 at 18:02
add a comment |
2
This actually creates a new array rather than updating the current array. See my answer for more detail.
– Colin T Bowers
Nov 25 '18 at 0:39
Yeah fair enough, your answer is better
– chasmani
Nov 26 '18 at 18:02
2
2
This actually creates a new array rather than updating the current array. See my answer for more detail.
– Colin T Bowers
Nov 25 '18 at 0:39
This actually creates a new array rather than updating the current array. See my answer for more detail.
– Colin T Bowers
Nov 25 '18 at 0:39
Yeah fair enough, your answer is better
– chasmani
Nov 26 '18 at 18:02
Yeah fair enough, your answer is better
– chasmani
Nov 26 '18 at 18:02
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53463436%2fhow-to-replace-inf-values-with-nan-in-array-in-julia-1-0%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown