Virtual Mobile 'HTTP-POST' Notification Not Working?

WebSMS and 'Email to SMS' services

Virtual Mobile 'HTTP-POST' Notification Not Working?

Postby trk on Mon Oct 11, 2010 9:07 pm

G'day,

I've got a virtual mobile that I was hoping to use the URL HTTP-POST notification on, but it doesnt seem to be doing the actual POST?

I'm currently tailing the access.log and can see myself GET'ing the php file I want to use, and have made a temporary page which is just a form that uses POST to that same PHP and can see that form submitting POST requests, but I cant see any access from any other Exetel IP/hostname except mine.

I am receiving the notification emails just fine, and it appears in the virtual mobile web interface just fine.

Does the POST component not work? Or am I doing something wrong?
trk
 
Posts: 59
Joined: Fri Feb 12, 2010 12:38 am
Location: Brisbane

Re: Virtual Mobile 'HTTP-POST' Notification Not Working?

Postby William M on Tue Oct 12, 2010 8:58 am

Hi,

Yes it does work.
I suggest you email residentialsupport@exetel.com.au with your details for more information.
William M
 
Posts: 739
Joined: Wed Mar 21, 2007 1:28 pm

Re: Virtual Mobile 'HTTP-POST' Notification Not Working?

Postby trk on Thu Oct 14, 2010 12:15 am

After a bit of fiddling around I worked it out. Dont put any 'http://' at the start ^_^

Interesting side note, if you have an Apache server running a redirect (for example to bring everyone to http://www.domain.com who arrives at domain.com), make sure you use the final destination instead of the version that redirects. Apache doesnt seem to carry over POST data when it redirects - or at least my hosts setup doesnt :shock:

Perhaps the 'Virtual Mobile' section in the members section could have a bit more info? I dont do much... anything... to do with online stuff so maybe its perfectly obvious to people who usually use these sort of things but just something basic like not having the http:// at the start could have saved me from scratching my head a bit.
trk
 
Posts: 59
Joined: Fri Feb 12, 2010 12:38 am
Location: Brisbane

Re: Virtual Mobile 'HTTP-POST' Notification Not Working?

Postby trk on Sat Oct 16, 2010 12:11 am

I think there is a problem that hopefully isnt caused by my lack of ability....

When an SMS contains an '&' character, the POST does not appear to contain all the information.

For example, I sent this message:
Message This is & test of & symbol and star * symbol and caret ^ symbol

... but the URL it posts to sees:

Array
(
[msgid] => 183574
[receivedat] => 2010-10-15 23:55:17
[sender] => 61416905xxx
[vmn] => 61416905xxx
[message] => This is
)


However, if I send the exact same data using a form on my site, submitted to the exact same URL (and just fill in the same details for the same variable names), I get the expected result:

Array
(
[msgid] => 183574
[receivedat] => 2010-10-15 23:55:17
[sender] => 61416905xxx
[vmn] => 61416905xxx
[message] => This is & test of & symbol and star * symbol and caret ^ symbol
)

(obviously I've edited the mobile number, but have left the msgid in case that helps with bug tracking or confirmation of anything).

I've sent an email to residentialsupport@exetel.com.au already, but I thought you (or someone else) may have an idea of why this would be. I can tell you the URL of the log file its outputting to so you can send test POST requests if that helps too?
trk
 
Posts: 59
Joined: Fri Feb 12, 2010 12:38 am
Location: Brisbane

Re: Virtual Mobile 'HTTP-POST' Notification Not Working?

Postby CoreyPlover on Sat Oct 16, 2010 10:38 pm

Have you "escaped" the punctuation / message string? "&" is a special character, used in HTML and delimits variables in querystrings.

"This is & test of & symbol and star * symbol and caret ^ symbol" would become "This is %26 test of %26 symbol and star * symbol and caret ^ symbol"
I am a volunteer moderator and not an Exetel staff member. As with all forum posts, mine do not constitute any "official" Exetel position. Support tickets may be logged via https://helpdesk.exetel.com.au or residentialsupport@exetel.com.au
User avatar
CoreyPlover
Volunteer Site Admin
 
Posts: 7075
Joined: Sat Nov 04, 2006 2:24 pm
Location: Melbourne, VIC

Re: Virtual Mobile 'HTTP-POST' Notification Not Working?

Postby trk on Sun Oct 17, 2010 3:30 am

CoreyPlover wrote:Have you "escaped" the punctuation / message string? "&" is a special character, used in HTML and delimits variables in querystrings.

"This is & test of & symbol and star * symbol and caret ^ symbol" would become "This is %26 test of %26 symbol and star * symbol and caret ^ symbol"


I dont receive the full message to be able to escape it in the first place :)

If I send this exact text "This is & test of & symbol and star * symbol and caret ^ symbol" via an SMS, either from my phone or from Exetels web SMS service, the PHP file I have specified receives the following $_POST data:

Code: Select all
Array
(
[msgid] => 183574
[receivedat] => 2010-10-15 23:55:17
[sender] => 61416905xxx
[vmn] => 61416905xxx
[message] => This is
)


which is the result of this code (in test.php):
Code: Select all
<?php

ob_start();
print_r($_POST);
$output = ob_get_clean();
file_put_contents('log.txt', file_get_contents('log.txt').$output);

?>



If I POST the exact same variables that Exetel uses to that PHP file using the following basic form instead of actually sending an SMS (thus bypassing whatever processing Exetel does to the message):
Code: Select all
<form action="test.php" method="post">
msgid: <input name="msgid"><br />
receivedat: <input name="receivedat"><br />
sender: <input name="sender"><br />
vmn: <input name="vmn"><br />
message: <input name="message"><br />
<input type="submit" value="submit"><br />
</form>


.... then I get the full message, aka
Code: Select all
Array
(
[msgid] => 183574
[receivedat] => 2010-10-15 23:55:17
[sender] => 61416905xxx
[vmn] => 61416905xxx
[message] => This is & test of & symbol and star * symbol and caret ^ symbol
)


Another example is this, received tonight:

The email received contains:
"Message:
Considering your livid hatred for the iPad, what do you think of the galaxy tab & other similar tablets?"

But the $_POST data contains:
" [message] => Considering your livid hatred for the iPad, what do you think of the galaxy tab"

A further example, this time with a whole heap of special characters that *ARENT* and ampersand:

The email received contains:
"Message:
1P40n35 R £337 ¡0¡z.... 73h h473????!?!??"

And the post also contains (a slightly mangled):
" [message] => 1P40n35 R ã337 á0áz.... 73h h473????!?!??"
trk
 
Posts: 59
Joined: Fri Feb 12, 2010 12:38 am
Location: Brisbane

Re: Virtual Mobile 'HTTP-POST' Notification Not Working?

Postby Dazzled on Sun Oct 17, 2010 9:25 am

It looks like the ampersand etc are interpreted in the POST. A Bash approach could be to use curl with the --data-urlencode argument to do the encoding. You can place the whole message in a variable, using backslashes if quotation marks are desired.

PS see urlencode and htmlspecialchars in PHP.
User avatar
Dazzled
Volunteer Site Admin
 
Posts: 6525
Joined: Mon Nov 13, 2006 1:16 pm
Location: Sydney

Re: Virtual Mobile 'HTTP-POST' Notification Not Working?

Postby trk on Sun Oct 17, 2010 8:07 pm

Dazzled wrote:It looks like the ampersand etc are interpreted in the POST. A Bash approach could be to use curl with the --data-urlencode argument to do the encoding. You can place the whole message in a variable, using backslashes if quotation marks are desired.

PS see urlencode and htmlspecialchars in PHP.


I have no trouble with the data once it arrives on my end though.... Using my own form to send data with the exact same variable names that Exetel supplies and it works perfectly. Rely on the data sent by Exetel after an SMS arrives and its missing everything from (and including) the first ampersand.

Is anyone else using the POST notification and receiving the ampersand symbols (and subsequent text) ok? That would prove its not Exetel at least.
trk
 
Posts: 59
Joined: Fri Feb 12, 2010 12:38 am
Location: Brisbane

Re: Virtual Mobile 'HTTP-POST' Notification Not Working?

Postby trk on Sun Oct 17, 2010 9:01 pm

For further proof, I created a post form on a totally different host altogether (its even in a different country ;)) and submitted that and the test.php file happily receives the full message, ampersand and all.

It has to be the Exetel script that submits the POST notification, right? :?
trk
 
Posts: 59
Joined: Fri Feb 12, 2010 12:38 am
Location: Brisbane

Re: Virtual Mobile 'HTTP-POST' Notification Not Working?

Postby Dazzled on Sun Oct 17, 2010 9:04 pm

Do you know how the Exetel script is coded? Ampersand is the data delimiter in url encoded pairs - are the inputs plain or escaped?

If it's done in the shell, there may (should?) be local IFS='&' with *$ being used to break up the whole string into name=values, eg vars="&$*&". If these array strings are then parsed to recognise names, and split from the '=', the value of after an unescaped '&' would be truncated nicely, while not upsetting the name=value structure.
User avatar
Dazzled
Volunteer Site Admin
 
Posts: 6525
Joined: Mon Nov 13, 2006 1:16 pm
Location: Sydney

Re: Virtual Mobile 'HTTP-POST' Notification Not Working?

Postby trk on Sun Oct 17, 2010 9:25 pm

Dazzled wrote:Do you know how the Exetel script is coded?


I wouldnt have a clue, I guess its why I'm asking here :P

All I've really got to go on is my access.log entry:
Code: Select all
220.233.0.46 - - [17/Oct/2010:03:16:01 -0700] "POST /post_sms.php HTTP/1.1" 200 168 "-" "-"


And the bit of code I wrote to capture the $_POST array in its entirety to a file (the results of which are the [ code ] blocks I've posted above a few times now, so I wont do it again)

Dazzled wrote:Ampersand is the data delimiter in url encoded pairs - are the inputs plain or escaped?


I assume this is what the problem is.... The script that is POST'ing the data is breaking the message component (which is received as $_POST['message']) at the ampersand. The reason I used print_r to capture the entire $_POST array was that I thought perhaps they were sending, for e.g. "Hello my name is trk & this is my message" with $_POST['message'] == "Hello my name is trk " and then $_POST['this'] == "is my message", if you know what I mean? But turns out not. Perhaps it is numeric and they send the first 'x' variables... Because the & breaks it and generates a new variable which is AFTER 'x' it never gets sent?

Dazzled wrote:If it's done in the shell, there may (should?) be local IFS='&' with *$ being used to break up the whole string into name=values, eg vars="&$*&". If these array strings are then parsed to recognise names, and split from the '=', the value of after an unescaped '&' would be truncated nicely, while not upsetting the name=value structure.


My only coding experience, which is incredibly limited (in case you cant tell from the undoubtedly many incorrect uses of terminology in this thread), is with PHP and HTML so I'm lost at this point :(


If this was PHP I'd be splitting the full data string at the that was about to be sent into chunks seperated by '&', then using them in numerical order (0 = msgid, 1=receivedat, 2=sender, 3=vmn as you know they cannot possibly contain the ampersand due to their data types) and then every other chunk combined for the 4th variable which is the message component. Or something. I am a tradie, not a computer guy ;)
trk
 
Posts: 59
Joined: Fri Feb 12, 2010 12:38 am
Location: Brisbane

Re: Virtual Mobile 'HTTP-POST' Notification Not Working?

Postby Dazzled on Sun Oct 17, 2010 9:52 pm

I've never seen the Exetel script either, so am only guessing, but the method I mentioned is an obvious approach to coding a problem like this in the shell. The power of the Unix/Linux shell (imagine Windows command prompt combined with BAT but a universe more advanced) is such that application programming is often not necessary. In this case it is being fed a single long POST input string, in the form
Code: Select all
username=xxxxxxxx&password=xxxxxxxx&mobilenumber=xxxxxxxx&message=xxxxxxxx&sender=xxxxxxxxx&messagetype=Text&referencenumber=xxxxxx


Now the shell has no data types - it is just a stream of characters. IFS is the shell's Internal Field Separator, so that text input can be automatically split into tokens - ie, lines of typed input usually are broken at the default separators, space, tab, newline. Script writers frequently change the IFS value(s) to simplify special parsing tasks. If we know that POST ampersand input is normally escaped, and has name=value fields separated by &, all we have to do is change IFS to & and we have your whole input instantly split into an array of name=xxxxxxx values. It is simple then for the script to split the desired names and their values, and discard any meaningless stuff. A side effect is truncation at any included &. (the truncated part lacks a meaningful name=)
User avatar
Dazzled
Volunteer Site Admin
 
Posts: 6525
Joined: Mon Nov 13, 2006 1:16 pm
Location: Sydney

Re: Virtual Mobile 'HTTP-POST' Notification Not Working?

Postby trk on Sun Oct 17, 2010 10:17 pm

Nice description, cheers.

So could changing the ISF to a character not accessible in the SMS standard (or at least, rarely used) be a simple solution to this? Though I'm sure there would be a knock-on effect.

Also I just found that I cant put a voicemail message on my virtual mobile to alert people who try to ring it that its not monitored :( These two items combined means I will probably end up shifting it to someone else anyway.... What a bummer, I try to keep all my & my workplaces services with Exetel because generally they're great.
trk
 
Posts: 59
Joined: Fri Feb 12, 2010 12:38 am
Location: Brisbane

Re: Virtual Mobile 'HTTP-POST' Notification Not Working?

Postby Dazzled on Sun Oct 17, 2010 10:25 pm

Did William M shed any light on this for you? Obviously I'm only saying how I'd have attempted it, based on the POST standards. IFS can actually temporarily be anything, and often is, but the standards already state the separator to be used, in a URL encoded string.

PS Exetel possibly has: htmlspecialchars($_POST["message"]). This will separate if you don't urlencode.
User avatar
Dazzled
Volunteer Site Admin
 
Posts: 6525
Joined: Mon Nov 13, 2006 1:16 pm
Location: Sydney

Re: Virtual Mobile 'HTTP-POST' Notification Not Working?

Postby trk on Sun Oct 17, 2010 10:27 pm

I havent had a reply to the email / support ticket raised for this, although the one I sent regarding the voicemail was replied to very quickly (in usual Exetel fashion).

I'm guessing the other one is just working its way through the chain until it finds someone who can answer it :P
trk
 
Posts: 59
Joined: Fri Feb 12, 2010 12:38 am
Location: Brisbane

Next

Return to SMS via ADSL

Who is online

Users browsing this forum: No registered users and 2 guests

cron