Trying to POST protobuf data with JMeter

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Trying to POST protobuf data with JMeter

Britt Cagnina
I have a Sampler HTTP Request setup with the body as:

b''

This is a simple empty protobuf object, but JMeter is actually sending some
wrapped version of this:

b"b''"

I have used Content-Type to be both application/protobuf and
application/octet-stream.

Is there a way to get JMeter to post EXACTLY what I give it?


I have an open stack overflow issue for better formatting:
https://stackoverflow.com/questions/50957492/jmeter-not-posting-binary-data-correctly

Thanks,
Britt
Reply | Threaded
Open this post in threaded view
|

Re: Trying to POST protobuf data with JMeter

daluu
I'm unaware of details of protobuf particularly with HTTP, but if you were
to make an HTTP request using a REST/API client/tool (or custom scripting)
or a browser based REST client, what would it actually look like for what
you send, and/or rather what does the raw data look like when you sniff it
say with an HTTP proxy like fiddler/charles.

I'm just wondering if jmeter is the only tool with the issue or other REST
clients have it too or not.

How are you POSTing the data exactly? Pasting the value in the GUI field in
the sampler? Or via a file upload/POST? If the former, I suggest you try
the latter of POSTing/uploading an empty protobuf encoded file with jmeter
and see if that works for you. And be sure the file is generated/encoded
using protobuf tooling (or scripts), not some text editor. Because protobuf
is a binary format as I recall, denoting it as b'' might just be treated as
plain text by jmeter regardless of the Content-Type (unless jmeter was
smart to key off that header value). A library that deals with protobuf or
some languages like python might understand b'' to be empty binary data,
but other tools may not and think that just a literal string value. Using
true protobuf encoded file (or dumping that value out) will ensure you get
the right encoding.

If the GUI based HTTP sampler isn't ideal or working, you could try the
scripting based samplers using Java, Groovy, Javascript together with
protobuf and HTTP libraries for those languages to do what you need.

On Wed, Jun 20, 2018 at 5:34 PM, Britt Cagnina <[hidden email]> wrote:

> I have a Sampler HTTP Request setup with the body as:
>
> b''
>
> This is a simple empty protobuf object, but JMeter is actually sending some
> wrapped version of this:
>
> b"b''"
>
> I have used Content-Type to be both application/protobuf and
> application/octet-stream.
>
> Is there a way to get JMeter to post EXACTLY what I give it?
>
>
> I have an open stack overflow issue for better formatting:
> https://stackoverflow.com/questions/50957492/jmeter-not-
> posting-binary-data-correctly
>
> Thanks,
> Britt
>
Reply | Threaded
Open this post in threaded view
|

Re: Trying to POST protobuf data with JMeter

Britt Cagnina
Thanks for the quick reply.

My application essential builds the protobuf object from the incoming wire
string. It is literally something like MyProtobufObject.fromString(...):

The POST payload typically looks something like this
b'\n\xb1\t\n$9c4a0f64-791e-4286-b4a3-ba862e4b31bc\x12\xa4\x01\n\x011\x12\x1f\x08\xac\x02\x10\xfa\x010\x010\x020\x060\x070\n0\x0b0\x0e\xc2>\x08\x1a\x06\x08\xac\x02\x10\xfa\x01Zr\x08\x00\x12\x1a\n'

Notice the single slashes.  I am able to use postman to post the exact body
content, which makes it to my backend sever intact and unaltered--but maybe
postman is just too kind.

However, when I am using JMeter, and now Apache Bench, the content that is
being sent not only has the added b'' wrapper, but also the backslashes are
backslashed again--which causes my issue.  I've tried the file upload
approach but am still getting this bad behavior.  It's the backslashes that
are getting me--as I could pass only
\n\xb1\t\n$9c4a0f64-791e-4286-b4a3-ba862e4b31bc\x12\xa4\x01\n\x011\x12\x1f\x08\xac\x02\x10\xfa\x010\x010\x020\x060\x070\n0\x0b0\x0e\xc2>\x08\x1a\x06\x08\xac\x02\x10\xfa\x01Zr\x08\x00\x12\x1a\n,
and have that added wrapper of b"" work in my favor, but getting the
following isn't good:

b'\\n\\xb1\\t\\n$9c4a0f64-791e-4286-b4a3-ba862e4b31bc\\x12\\xa4\\x01\\n\\x011\\x12\\x1f\\x08\\xac\\x02\\x10\\xfa\\x010\\x010\\x020\\x060\\x070\\n0\\x0b0\\x0e\\xc2>\\x08\\x1a\\x06\\x08\\xac\\x02\\x10\\xfa\\x01Zr\\x08\\x00\\x12\\x1a\\n'


On Wed, Jun 20, 2018 at 6:15 PM, David Luu <[hidden email]> wrote:

> I'm unaware of details of protobuf particularly with HTTP, but if you were
> to make an HTTP request using a REST/API client/tool (or custom scripting)
> or a browser based REST client, what would it actually look like for what
> you send, and/or rather what does the raw data look like when you sniff it
> say with an HTTP proxy like fiddler/charles.
>
> I'm just wondering if jmeter is the only tool with the issue or other REST
> clients have it too or not.
>
> How are you POSTing the data exactly? Pasting the value in the GUI field in
> the sampler? Or via a file upload/POST? If the former, I suggest you try
> the latter of POSTing/uploading an empty protobuf encoded file with jmeter
> and see if that works for you. And be sure the file is generated/encoded
> using protobuf tooling (or scripts), not some text editor. Because protobuf
> is a binary format as I recall, denoting it as b'' might just be treated as
> plain text by jmeter regardless of the Content-Type (unless jmeter was
> smart to key off that header value). A library that deals with protobuf or
> some languages like python might understand b'' to be empty binary data,
> but other tools may not and think that just a literal string value. Using
> true protobuf encoded file (or dumping that value out) will ensure you get
> the right encoding.
>
> If the GUI based HTTP sampler isn't ideal or working, you could try the
> scripting based samplers using Java, Groovy, Javascript together with
> protobuf and HTTP libraries for those languages to do what you need.
>
> On Wed, Jun 20, 2018 at 5:34 PM, Britt Cagnina <[hidden email]> wrote:
>
> > I have a Sampler HTTP Request setup with the body as:
> >
> > b''
> >
> > This is a simple empty protobuf object, but JMeter is actually sending
> some
> > wrapped version of this:
> >
> > b"b''"
> >
> > I have used Content-Type to be both application/protobuf and
> > application/octet-stream.
> >
> > Is there a way to get JMeter to post EXACTLY what I give it?
> >
> >
> > I have an open stack overflow issue for better formatting:
> > https://stackoverflow.com/questions/50957492/jmeter-not-
> > posting-binary-data-correctly
> >
> > Thanks,
> > Britt
> >
>