How To Create an RSS Aggregator with PHP and AJAX
Date: Dec 9, 2005
For more information on Web design, visit our Web Design Reference Guide or sign up for our Web Design Newsletter.
Imagine using a simple HTML file to send a request to a server-side script, receive a custom XML file based on that request, and then display it to the user—without ever refreshing the browser! This article explains how to use a combination of PHP and AJAX to create real-time data transfers in any application without a browser refresh.
Although this article uses PHP, keep in mind that any server-side language will work. I’ll assume that you have a basic understanding of JavaScript and PHP or a similar server-side language. I created a sample project that can be viewed here, and the source code from the sample can be downloaded here to help you get started.
The sample uses AJAX to send a request for an RSS feed to a custom PHP object. The PHP object makes a copy of the feed on the local server and returns the path. The requesting object receives the path, parses it, and displays the data as HTML to the user. It sounds like a lot of steps, but it consists of only four small files, which leverage their own specific strengths and make the entire process extremely efficient.
I’m sure that some of you are asking why you would create a copy of the feed on the local server instead of simply parsing the original feed. The reasoning behind this approach is to allow the bypassing of cross-domain restrictions that the XML HTTP Request object imposes. I’ll explain how to create this custom PHP object, but first let’s get started with the form.
Creating the Form To Make the Request
The first thing to do is include the JavaScript and any CSS files that you may want to use between the head tags in your HTML. I included one stylesheet to take care of the finished layout for the aggregator and the JavaScript file that we’ll create to make requests and parse feeds:
<link href="css/layout.css" rel="stylesheet" type="text/css" /> <script src="js/request.js"></script>
Next, create a form that sends a request for an RSS feed of your choice. The form I created simply includes an input field and a button to submit the request. The query for the request is a string composed of the feed input value and a password that will be verified on the server side; as an example, I used this:
"password=mypassword
The example also makes a request each time the page is loaded; therefore, if the page is refreshed, the existing feed string in the input field will be requested when the page loads. Following is a sample of the form data, along with a few div tags for displaying specific nodes in the parsed feed:
<body onload="javascript:makeRequest(’request.php?request=’ + document.feedForm.feed.value + ’"password=mypassword’);"> <form name="feedForm" method="post" action="javascript:makeRequest(’request.php?request=’ + document.feedForm.feed.value + ’"password=mypassword’);"> Enter a feed: <input type="text" name="feed" id="feed" size="20"> <input type="submit" name="submit" value="Add Feed"> </form> <div id="logo"></div> <hr/> <div id="copy"></div> <div id="details"></div> </body>
The three div tags that I created are logo, copy, and details, each of which has a style associated with it in the layout stylesheet. These will be used when we’ve parsed the feed, but we first need to be able to access the feed that we’re requesting. This will be accomplished with the PHP object that I mentioned earlier.
Creating the Custom PHP Object
I created a small RSS class with PHP, which creates a copy of a requested feed on the local server so that it can be accessed by the XML HTTP Request object that we’ll create shortly. Typically, you can’t request a file cross-domain, meaning that the file that you’re requesting needs to be located on the local server. This class is a workaround to the cross-domain issue because it creates a copy of the feed that was requested on the local server and returns the local path to the feed, which is then accessible by the Request object.
The only method in the class is a request method, which takes one parameter, the URL to the requested RSS feed. It then checks whether a directory by the name of rss is located on the local server. If not, it creates one and sets the permission mode to 0666, which means that the directory is readable and writeable. It’s set to readable so that the directory can be accessible at a later time and set to writeable to allow a copy of the feed to be written to the directory on the local server:
// Make directory if one does not exist
$dir = "rss";
if(!is_dir($dir))
{
mkdir($dir, 0666);
}
Before copying the feed to the server we need a unique filename. I’m using the md5 encryption method on the entire URL to ensure that the names of all feeds are unique. With this new filename, concatenate a string that represents the path to the file; this will be used when creating the copy of the feed:
// Create unique names $file = md5($rss_url); $path = "$dir/$file.xml";
Using the path that was defined above and the reference to the original URL of the requested feed, we can now create a copy of the file. Finally, return the path to the new file as the response to the request:
// Copy feed to local server copy($rss_url, "$path"); return $path;
Following is the small, yet powerful RSS class in its entirety:
<?php
class RSS
{
function get($rss_url)
{
if($rss_url != "")
{
// Make directory if one does not exist
$dir = "rss";
if(!is_dir($dir))
{
mkdir($dir, 0666);
}
// Create unique names
$file = md5($rss_url);
$path = "$dir/$file.xml";
// Copy feed to local server
copy($rss_url, "$path");
return $path;
}
}
}
?>
In order to access the request method in the PHP class, there’s a request file that acts as an interface to the class, which is the file that we’ll be requesting. This file first verifies a password variable from the requesting query and either returns a message stating that the requester is not an authorized user or responds with the path to the RSS feed that was copied to the local server after it has been processed by the request method. To respond with the RSS feed, the RSS object needs to be included and instantiated, and the request method needs to be triggered with the URL of the feed requested as a parameter:
<?
if($password == "mypassword")
{
require_once(’classes/RSS.class.php’);
$rss = new RSS();
echo $rss->get($request);
}
else
{
echo "You are an unauthorized user";
}
?>
GET and POST with AJAX
To POST the request, we first need to create the request object. If you don’t have prior experience creating the request object, read my article "How To Use AJAX" or simply study the sample source code. Once the request object is created, call the sendFeed method and pass the URL that was created by the form:
function sendFeed(url)
{
post.onreadystatechange = sendRequest;
post.open("POST", url, true);
post.send(url);
}
Once the response from the PHP object is received and properly loaded, make another request on the local file with which it responded. In this case, post.responseText provides us with the path to the new file:
function sendRequest()
{
if(checkReadyState(post))
{
request = createRequestObject();
request.onreadystatechange = onResponse;
request.open("GET", post.responseText, true);
request.send(null);
}
}
Parsing the Response
Parsing the response is a bit more challenging because of the differences in RSS feeds. Some have images, which include a title and description node, and some don’t. Therefore, when we parse the feed we need to do a little checking to decipher whether it includes an image. If it does, I display the image in the image div tag, along with the title and link for the feed:
var _logo = "";
var _title = response.getElementsByTagName(’title’)[0].firstChild.data;
var _link = response.getElementsByTagName(’link’)[0].firstChild.data;;
_logo += "<a href=’" + _link + "’ target=’_blank’>" + _title + "</a><br/>";
if(checkForTag(response.getElementsByTagName(’image’)[0]))
{
var _url = response.getElementsByTagName(’url’)[0].firstChild.data;
_logo += "<img src=’" + _url + "’ border=’0’><br/>"
}
document.getElementById(’logo’).innerHTML = _logo;
Not only do we have to check for an image to display it, we also need to check when we’re looping through all of the items in the feed, because if there’s an image all the other title and link node indexes get thrown out of whack. Therefore, when the image tag is found, we adjust the index for the title and link nodes by incrementing the index by one with each pass:
if(checkForTag(response.getElementsByTagName(’image’)[0]) "" i>0)
{
var _title = response.getElementsByTagName(’title’)[i+1].firstChild.data;
var _link = response.getElementsByTagName(’link’)[i+1].firstChild.data;
}
else
{
var _title = response.getElementsByTagName(’title’)[i].firstChild.data;
var _link = response.getElementsByTagName(’link’)[i].firstChild.data;
}
You can use the checkForTag method to check whether specific tags exist:
function checkForTag(tag)
{
if(tag != undefined)
{
return true;
}
else
{
return false;
}
}
There are many possibilities for parsing a feed. For example, you can assign items to categories and make the categories collapsible so that users can choose what they want to see. As an example, I categorized items by date by deciphering whether the pubDate for a particular item was different from the previous item’s pubDate and displayed a new date accordingly:
if(i>1)
{
var previousPubDate = response.getElementsByTagName(’pubDate’)[i-1].firstChild.data;
}
if(pubDate != previousPubDate || previousPubDate == undefined)
{
_copy += "<div id=’detail’>" + pubDate + "</div><hr align=’left’ width=’90%’/>";
}
_copy += "<a href=\"javascript:showDetails(’" + i + "’);\">" + _title + "</a><br/><br/>";
document.getElementById(’copy’).innerHTML += _copy;
The last piece of functionality is the showDetails method, which is used to display details when a user selects a specific item from a feed. It takes one parameter, the item index, and uses it to find the index of the details node in the feed:
function showDetails(index)
{
document.getElementById(’details’).innerHTML = response.getElementsByTagName(’description’)[index].firstChild.data;
}
To view the entire JavaScript file, reference the source code for the project. This file also includes the methods that create the request object and handle loading, which I explain in detail in my article "How To Use AJAX."
What’s Next?
Using AJAX to send query strings to a server-side script and retrieving a custom response based on that query opens extraordinary possibilities to web developers. With this knowledge, your next web application will be full of new possibilities, but there’s still one thing missing: a database. In my next article, I’ll show you how simple it is to interact with a database by using AJAX and PHP.
For more information on Web design, visit our Web Design Reference Guide or sign up for our Web Design Newsletter.