Searching under different sites

[marvin]

Hi guys

 

Just wondering if anyone can help me on this!

 

Basically i was wondering whether anyone knew of a system that could basically search other sites from a user input on a different site? Something similar to price comparison sites that take an input from a user on a site, submits it to different sites or a single site, searches within that database and then feeds the information back to the user?

 

Hope this makes sense, any thought on this would be appreciated!

[howartp]

Hi Marvin,

 

Would something like DogPile meet your needs?

 

(Yes, I know it sounds inappropriate but believe me it's a legit site!)

 

DogPile takes your search term and re-submits it to others such as google and yahoo etc, then displays the results from each?

 

Peter

[marvin]

That would only access search engines...

What i want is my site to collect data from user, that data is stored on a database on my side, send it to a specific site that will check that search criteria on their side and then tell me the answer.

 

I know this can be done by the user going directly to that site but what i want is do some calculation from the result of that site and then tell the user about it!

 

Hope that makes sense.

[howartp]

Hope that makes sense.

Kind of...!

 

I think you'd need something like PHP's cURL library (or any other language with similar features) which will allow you to take an input ($query) then PHP will pop along to the target site (google for example) and pass it the $query (via the address bar) then be given back the answer (result page) that you would normally see, in a big long string being the html content of the results page.

 

You could then parse this and obtain the bits you want out of it, perform calculations, then output whatever to the user?

 

Peter

[marvin]

so for example let's assume

 

If i design a form which collects some data from the user such as Name, surname, travelling from -> to, time travelling and the date.

 

Now i want that information fed to let's say Book Flights, Hotels, Holidays, Car Rental with British Airways - BA.com so that it will check the flight available during the travel request dates and timings and then feed that information back to me on my site.

 

Is this possible? And how could i do this?

[howartp]

Is this possible?

Unlikely.

 

The site you want to search would ideally need to be one which uses the address bar to pass parameters. So for example with Google, which would work, you can go to:

 

http://www.google.co.uk/search?q=my_search_term

 

This would mean you could take the input yourself ('my_search_term') then pass it to Google.

 

However BA doesn't appear to have anything of the form:

 

www.ba.com/book?type=flight&from=Gatwick&to=Luton&depart=251209&return=030110&tickets=1

 

which you would need to be able to search results from another site.

 

If you can find a BA alternative which has something of the form shown for Google/BA, then you would be able to do it.

 

If you do find one, you'd have a normal html form on your page, with it's 'action' set to processform.php which would have something like the following in:

_

 

<?php

//Set variables to answers from form

$searchterm = $_POST["searchterm"];

 

// create curl resource

$ch = curl_init();

curl_setopt($ch, CURLOPT_URL, "www.google.co.uk/search?q=" $searchterm);

curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

$output = curl_exec($ch);

curl_close($ch);

 

?>

 

Don't worry about what all that does for the moment, but it will give you $output which will contain the html code of the results from google (or the BA alternative in your case) as follows:

 

Google Results



Result1
Result2
Result3

 

Unfortunately the html code will be far more complicated than that, but if you can parse that lot into a list of:

 

Result1 = BBC - Homepage

Result2 = Yahoo! UK & Ireland

 

(or, in your case:)

 

Flight1 = London to Gatwick, 25th December, 10.04am

Flight2 = London to Gatwick via Luton, 25th December, 11.26am

etc

 

then you can do what you like entering it in the Database and displaying results.

_

 

I don't know how much website coding you know, but hopefully if you know enough to be undertaking this project, you will be able to take what I've given and turn it into full code...?

 

Peter

Edited November 25, 2009 by howartp

[marvin]

I thought this was kind of not possible!

Basically the only way i could do this is if i had access to their live DB which i will never have access to?

[howartp]

I thought this was kind of not possible!

Basically the only way i could do this is if i had access to their live DB which i will never have access to?

 

For someone like BA, yes that would likely be the only way.

 

Google and other search engines can be done far easier, as above.

 

Peter