Content Length in Golang
I couldn't find anything helpful online on this one.
I am writing an REST API, and I want to log the size of the body of the request in bytes for metrics. Go net/http API does not provide that directly. http.Request does have Content-Length field, but that field can be empty or the client might send false data.
Is there a way to get that in the middlware level? The bruteforce method would be to read the full body and check the size. But if I do that in the middleware, the handler will not have access to the body because it would have been read and closed.
http
go asked Nov 20 at 22:32
Husain
427416
add a comment |
I couldn't find anything helpful online on this one.
I am writing an REST API, and I want to log the size of the body of the request in bytes for metrics. Go net/http API does not provide that directly. http.Request does have Content-Length field, but that field can be empty or the client might send false data.
Is there a way to get that in the middlware level? The bruteforce method would be to read the full body and check the size. But if I do that in the middleware, the handler will not have access to the body because it would have been read and closed.
http
go asked Nov 20 at 22:32
Husain
427416
It's still, as you say, "bruteforce", but you CAN use a middleware to read it, check the size, then replace it back into theRequest.Bodyusing a newio.ReadCloserbecause that is all theBodyis. golang.org/pkg/net/http/#Request
– RayfenWindspear
Nov 20 at 22:51
add a comment |
I couldn't find anything helpful online on this one.
I am writing an REST API, and I want to log the size of the body of the request in bytes for metrics. Go net/http API does not provide that directly. http.Request does have Content-Length field, but that field can be empty or the client might send false data.
Is there a way to get that in the middlware level? The bruteforce method would be to read the full body and check the size. But if I do that in the middleware, the handler will not have access to the body because it would have been read and closed.
http
go asked Nov 20 at 22:32
Husain
427416
I couldn't find anything helpful online on this one.
I am writing an REST API, and I want to log the size of the body of the request in bytes for metrics. Go net/http API does not provide that directly. http.Request does have Content-Length field, but that field can be empty or the client might send false data.
Is there a way to get that in the middlware level? The bruteforce method would be to read the full body and check the size. But if I do that in the middleware, the handler will not have access to the body because it would have been read and closed.
http
go http
go asked Nov 20 at 22:32
Husain
427416
asked Nov 20 at 22:32
Husain
427416
asked Nov 20 at 22:32
Husain
427416
asked Nov 20 at 22:32
Husain
427416
asked Nov 20 at 22:32
Husain
427416
427416
It's still, as you say, "bruteforce", but you CAN use a middleware to read it, check the size, then replace it back into theRequest.Bodyusing a newio.ReadCloserbecause that is all theBodyis. golang.org/pkg/net/http/#Request
– RayfenWindspear
Nov 20 at 22:51
add a comment |
It's still, as you say, "bruteforce", but you CAN use a middleware to read it, check the size, then replace it back into theRequest.Bodyusing a newio.ReadCloserbecause that is all theBodyis. golang.org/pkg/net/http/#Request
– RayfenWindspear
Nov 20 at 22:51
It's still, as you say, "bruteforce", but you CAN use a middleware to read it, check the size, then replace it back into the
Request.Body using a new io.ReadCloser because that is all the Body is. golang.org/pkg/net/http/#Request– RayfenWindspear
Nov 20 at 22:51
It's still, as you say, "bruteforce", but you CAN use a middleware to read it, check the size, then replace it back into the
Request.Body using a new io.ReadCloser because that is all the Body is. golang.org/pkg/net/http/#Request– RayfenWindspear
Nov 20 at 22:51
add a comment |
2 Answers
2
active
oldest
votes
Why do you want a middle in here?
The simple way is b, err = io.Copy(anyWriterOrMultiwriter, r.Body)b is total content length of request when err == nil
Use request body as you want. Also b, err = io.Copy(ioutil.Discard, r.Body)
answered Nov 21 at 2:14
KibGzr
1,466610
add a comment |
You could write a custom ReadCloser that proxies an existing one and counts bytes as it goes. Something like:
type LengthReader struct {
Source io.ReadCloser
Length int
}
func (r *LengthReader) Read(b byte) (int, error) {
n, err := r.Source.Read(b)
r.Length += n
return n, err
}
func (r *LengthReader) Close() error {
var buf [32]byte
var n int
var err error
for err == nil {
n, err = r.Source.Read(buf[:])
r.Length += n
}
closeerr := r.Source.Close()
if err != nil && err != io.EOF {
return err
}
return closeerr
}
This will count bytes as you read them from the stream, and when closed it will consume and count all remaining unread bytes first. After you're finished with the stream, you can then access the length.
answered Nov 20 at 23:03
DarthFennec
1,291711
Brilliant! Almost makes it seem odd that this isn't a builtin thing already. And here I was trying to figure out how to avoid copies using middleware.
– RayfenWindspear
Nov 20 at 23:08
This is a smart way of doing it indeed. The only issue (and i failed to mention this in the question) is that i would not know the size of the request when it comes, rather after it is read.
– Husain
Nov 21 at 13:14
1
@Husain Fair. Though it's impossible in the general case to get the size of the request without reading it first, and with the other answer you're simply reading the entire request up front and storing it to be reread later.
– DarthFennec
Nov 26 at 21:06
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53402582%2fcontent-length-in-golang%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
Why do you want a middle in here?
The simple way is b, err = io.Copy(anyWriterOrMultiwriter, r.Body)b is total content length of request when err == nil
Use request body as you want. Also b, err = io.Copy(ioutil.Discard, r.Body)
answered Nov 21 at 2:14
KibGzr
1,466610
add a comment |
Why do you want a middle in here?
The simple way is b, err = io.Copy(anyWriterOrMultiwriter, r.Body)b is total content length of request when err == nil
Use request body as you want. Also b, err = io.Copy(ioutil.Discard, r.Body)
answered Nov 21 at 2:14
KibGzr
1,466610
add a comment |
Why do you want a middle in here?
The simple way is b, err = io.Copy(anyWriterOrMultiwriter, r.Body)b is total content length of request when err == nil
Use request body as you want. Also b, err = io.Copy(ioutil.Discard, r.Body)
answered Nov 21 at 2:14
KibGzr
1,466610
Why do you want a middle in here?
The simple way is b, err = io.Copy(anyWriterOrMultiwriter, r.Body)b is total content length of request when err == nil
Use request body as you want. Also b, err = io.Copy(ioutil.Discard, r.Body)
answered Nov 21 at 2:14
KibGzr
1,466610
answered Nov 21 at 2:14
KibGzr
1,466610
answered Nov 21 at 2:14
KibGzr
1,466610
answered Nov 21 at 2:14
KibGzr
1,466610
1,466610
add a comment |
add a comment |
You could write a custom ReadCloser that proxies an existing one and counts bytes as it goes. Something like:
type LengthReader struct {
Source io.ReadCloser
Length int
}
func (r *LengthReader) Read(b byte) (int, error) {
n, err := r.Source.Read(b)
r.Length += n
return n, err
}
func (r *LengthReader) Close() error {
var buf [32]byte
var n int
var err error
for err == nil {
n, err = r.Source.Read(buf[:])
r.Length += n
}
closeerr := r.Source.Close()
if err != nil && err != io.EOF {
return err
}
return closeerr
}
This will count bytes as you read them from the stream, and when closed it will consume and count all remaining unread bytes first. After you're finished with the stream, you can then access the length.
answered Nov 20 at 23:03
DarthFennec
1,291711
Brilliant! Almost makes it seem odd that this isn't a builtin thing already. And here I was trying to figure out how to avoid copies using middleware.
– RayfenWindspear
Nov 20 at 23:08
This is a smart way of doing it indeed. The only issue (and i failed to mention this in the question) is that i would not know the size of the request when it comes, rather after it is read.
– Husain
Nov 21 at 13:14
1
@Husain Fair. Though it's impossible in the general case to get the size of the request without reading it first, and with the other answer you're simply reading the entire request up front and storing it to be reread later.
– DarthFennec
Nov 26 at 21:06
add a comment |
You could write a custom ReadCloser that proxies an existing one and counts bytes as it goes. Something like:
type LengthReader struct {
Source io.ReadCloser
Length int
}
func (r *LengthReader) Read(b byte) (int, error) {
n, err := r.Source.Read(b)
r.Length += n
return n, err
}
func (r *LengthReader) Close() error {
var buf [32]byte
var n int
var err error
for err == nil {
n, err = r.Source.Read(buf[:])
r.Length += n
}
closeerr := r.Source.Close()
if err != nil && err != io.EOF {
return err
}
return closeerr
}
This will count bytes as you read them from the stream, and when closed it will consume and count all remaining unread bytes first. After you're finished with the stream, you can then access the length.
answered Nov 20 at 23:03
DarthFennec
1,291711
Brilliant! Almost makes it seem odd that this isn't a builtin thing already. And here I was trying to figure out how to avoid copies using middleware.
– RayfenWindspear
Nov 20 at 23:08
This is a smart way of doing it indeed. The only issue (and i failed to mention this in the question) is that i would not know the size of the request when it comes, rather after it is read.
– Husain
Nov 21 at 13:14
1
@Husain Fair. Though it's impossible in the general case to get the size of the request without reading it first, and with the other answer you're simply reading the entire request up front and storing it to be reread later.
– DarthFennec
Nov 26 at 21:06
add a comment |
You could write a custom ReadCloser that proxies an existing one and counts bytes as it goes. Something like:
type LengthReader struct {
Source io.ReadCloser
Length int
}
func (r *LengthReader) Read(b byte) (int, error) {
n, err := r.Source.Read(b)
r.Length += n
return n, err
}
func (r *LengthReader) Close() error {
var buf [32]byte
var n int
var err error
for err == nil {
n, err = r.Source.Read(buf[:])
r.Length += n
}
closeerr := r.Source.Close()
if err != nil && err != io.EOF {
return err
}
return closeerr
}
This will count bytes as you read them from the stream, and when closed it will consume and count all remaining unread bytes first. After you're finished with the stream, you can then access the length.
answered Nov 20 at 23:03
DarthFennec
1,291711
You could write a custom ReadCloser that proxies an existing one and counts bytes as it goes. Something like:
type LengthReader struct {
Source io.ReadCloser
Length int
}
func (r *LengthReader) Read(b byte) (int, error) {
n, err := r.Source.Read(b)
r.Length += n
return n, err
}
func (r *LengthReader) Close() error {
var buf [32]byte
var n int
var err error
for err == nil {
n, err = r.Source.Read(buf[:])
r.Length += n
}
closeerr := r.Source.Close()
if err != nil && err != io.EOF {
return err
}
return closeerr
}
This will count bytes as you read them from the stream, and when closed it will consume and count all remaining unread bytes first. After you're finished with the stream, you can then access the length.
answered Nov 20 at 23:03
DarthFennec
1,291711
answered Nov 20 at 23:03
DarthFennec
1,291711
answered Nov 20 at 23:03
DarthFennec
1,291711
answered Nov 20 at 23:03
DarthFennec
1,291711
1,291711
Brilliant! Almost makes it seem odd that this isn't a builtin thing already. And here I was trying to figure out how to avoid copies using middleware.
– RayfenWindspear
Nov 20 at 23:08
This is a smart way of doing it indeed. The only issue (and i failed to mention this in the question) is that i would not know the size of the request when it comes, rather after it is read.
– Husain
Nov 21 at 13:14
1
@Husain Fair. Though it's impossible in the general case to get the size of the request without reading it first, and with the other answer you're simply reading the entire request up front and storing it to be reread later.
– DarthFennec
Nov 26 at 21:06
add a comment |
Brilliant! Almost makes it seem odd that this isn't a builtin thing already. And here I was trying to figure out how to avoid copies using middleware.
– RayfenWindspear
Nov 20 at 23:08
This is a smart way of doing it indeed. The only issue (and i failed to mention this in the question) is that i would not know the size of the request when it comes, rather after it is read.
– Husain
Nov 21 at 13:14
1
@Husain Fair. Though it's impossible in the general case to get the size of the request without reading it first, and with the other answer you're simply reading the entire request up front and storing it to be reread later.
– DarthFennec
Nov 26 at 21:06
Brilliant! Almost makes it seem odd that this isn't a builtin thing already. And here I was trying to figure out how to avoid copies using middleware.
– RayfenWindspear
Nov 20 at 23:08
Brilliant! Almost makes it seem odd that this isn't a builtin thing already. And here I was trying to figure out how to avoid copies using middleware.
– RayfenWindspear
Nov 20 at 23:08
This is a smart way of doing it indeed. The only issue (and i failed to mention this in the question) is that i would not know the size of the request when it comes, rather after it is read.
– Husain
Nov 21 at 13:14
This is a smart way of doing it indeed. The only issue (and i failed to mention this in the question) is that i would not know the size of the request when it comes, rather after it is read.
– Husain
Nov 21 at 13:14
1
1
@Husain Fair. Though it's impossible in the general case to get the size of the request without reading it first, and with the other answer you're simply reading the entire request up front and storing it to be reread later.
– DarthFennec
Nov 26 at 21:06
@Husain Fair. Though it's impossible in the general case to get the size of the request without reading it first, and with the other answer you're simply reading the entire request up front and storing it to be reread later.
– DarthFennec
Nov 26 at 21:06
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53402582%2fcontent-length-in-golang%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
It's still, as you say, "bruteforce", but you CAN use a middleware to read it, check the size, then replace it back into the
Request.Bodyusing a newio.ReadCloserbecause that is all theBodyis. golang.org/pkg/net/http/#Request– RayfenWindspear
Nov 20 at 22:51