Parsing Amazon RSS problem with "title" tag.

[Matt Partner]

Hello to everyone,

First of all I want to say how useful has been in the past this forum to me and thank to all people who collaborates in this website.

Ok here we go with my problem. I'm trying to parse an Amazon RSS Feed like this:

Amazon product

The class in Java which I use is that:

package com.example.trovusearch;

import java.util.ArrayList;
import java.util.HashMap;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.NodeList;

import android.app.ListActivity;
import android.content.Intent;
import android.os.Bundle;
import android.os.StrictMode;
import android.util.Log;
import android.view.View;
import android.widget.AdapterView;
import android.widget.ListAdapter;
import android.widget.ListView;
import android.widget.SimpleAdapter;
import android.widget.TextView;
import android.widget.AdapterView.OnItemClickListener;
public class AmazonXmlParsingActivity extends ListActivity {
	//All static variables
	
	public static String txtSearch = new String();
	public void setTxtSearch(String text){
		txtSearch = text;
	}
	static String com = "http://www.amazon.com/rss/tag/";
	final String com2 = com.concat(txtSearch);
	final String URL_AMAZON = com2.concat("/new/");

	
	// XML node keys
	static final String KEY_ITEM = "item"; // parent node
	static final String KEY_ID = "id";
	static final String KEY_TITLE = "title"; // parent node
	static final String KEY_DESCRIPTION = "description";
	static final String KEY_LINK = "link";
	static final String KEY_IMG = new String();
	
	public void onCreate(Bundle savedInstanceState) {
		super.onCreate(savedInstanceState);
		if (android.os.Build.VERSION.SDK_INT > 9) {
		      StrictMode.ThreadPolicy policy = new StrictMode.ThreadPolicy.Builder().permitAll().build();
		      StrictMode.setThreadPolicy(policy);
		    }
		
		//This hashmap stores ID, TITLE and DESCRIPTION.
		ArrayList<HashMap<String, String>> menuItems = new ArrayList<HashMap<String, String>>();
		final TrovuXmlParser parser = new TrovuXmlParser();
		String xml = parser.getXmlFromUrl(URL_AMAZON); // getting XML
		Document doc = parser.getDomElement(xml); // getting DOM element
		NodeList nl = doc.getElementsByTagName(KEY_ITEM);
		final ArrayList<String> arrayImg = new ArrayList<String>();

		// looping through all item nodes <item>
			for (int i = 0; i < nl.getLength(); i++) {
				// creating new HashMap
				HashMap<String, String> map = new HashMap<String, String>();
				Element e = (Element) nl.item(i);
				// adding each child node to HashMap key => value
				map.put(KEY_ID, parser.getValue(e, KEY_ID));
				map.put(KEY_ITEM, parser.getValue(e, KEY_ITEM));
				map.put(KEY_TITLE, parser.getValue(e, KEY_TITLE));
				map.put(KEY_DESCRIPTION, parser.getValue(e, KEY_DESCRIPTION));
				map.put(KEY_LINK, parser.getValue(e, KEY_LINK));
				// adding HashList to ArrayList
				menuItems.add(map);
				Log.v("KEY", parser.getValue(e, KEY_DESCRIPTION));
				//adding imgUrl to a second arraylist.
				arrayImg.add(parser.getImg(e, KEY_DESCRIPTION));
			}
			// Adding menuItems to ListView
			ListAdapter adapter = new SimpleAdapter(this, menuItems,
					R.layout.list_item,
					new String[] {KEY_TITLE, KEY_DESCRIPTION, KEY_LINK}, new int[] {
							R.id.title, R.id.description, R.id.link_website});
			setListAdapter(adapter);
			
			// selecting single ListView item
			ListView lv = getListView();
			lv.setOnItemClickListener(new OnItemClickListener() {
				public void onItemClick(AdapterView<?> parent, View view,
						int position, long id) {
					// getting values from selected ListItem
					String title = ((TextView) view.findViewById(R.id.title)).getText().toString();
					String descFinal = ((TextView) view.findViewById(R.id.description)).getText().toString();
					String link = ((TextView) view.findViewById(R.id.link_website)).getText().toString();
					String imgUrl = arrayImg.get(position);
					Intent in = new Intent(getApplicationContext(), TrovuMenuItemActivity.class);
					in.putExtra(KEY_TITLE, title);
					in.putExtra(KEY_DESCRIPTION, descFinal);
					in.putExtra(KEY_IMG, imgUrl);
					in.putExtra(KEY_LINK, link);
					startActivity(in);
				}
			});
		}
}

I use this code in an Ebay Feed and works very good, but when I try to parse an Amazon RSS Feed the "title" tag appears in blank, only parses the description (And the photo of the product, because I wrote a class to manage it and works very well).

My problem and question is if anyone could help to me to see how I can to parse the "title" tag because doesn't work. Happens the same with the "link" tag but I think if I know how to parse the "title" I'll be able to fix the "link" problem.

Thanks to everyone and if you need more information about this, I'll post here.

Sincerelly,
Matt

[Steve Luke]

So what is the purpose of that TrovuXMLParser class, and where is the API. Since it is responsible for pulling the data out of the DOM we need to see how it works. I don't see what benefit it adds to the built-in tools that ship with Android (except that we all know the built-in ones and can access their documentation).

My suggestion would be to drop whatever the TrovuXMLParser is doing and use:

// Get a builder to create the document
javax.xml.parsers.DocumentBuilder documentBuilder = 
    javax.xml.parsers.DocumentBuilderFactory.newDocumentBuilder();

// Generate the document from the URI
Document amazonRSSFeed = documentBuilder.parse(URL_AMAZON);

// Get the items that are in the Channel
NodeList itemList = amazonRSSFeed.getDocumentElement().getElementsByName(KEY_ITEM);

//Loop through all items
int numberOfItems = itemList.getLength();
for (int currentItemNo = 0; currentItemNo < numberOfItems; currentItemNo++)
{
    Element currentItem = (Element)itemList.item(currentItemNo);

    // Get the title element
    Node titleNode = currentItem.getElementsByTagName(KEY_TITLE).item(0);
    // Get the value of the title's text node
    String title = titleNode.getFirstChild().getNodeValue();

    // repeat for other values you care about
}

[Matt Partner]

Thanks for your help Steve!

I edited the first post with the API I missed, I'm sorry but I'm so new in Java and I forgot a lot of things and how to comment correctly.

TrovuXmlParser class is the class I use to get the XML from the URL:

package com.example.trovusearch;

import java.io.IOException;
import java.io.StringReader;
import java.io.UnsupportedEncodingException;

import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import javax.xml.parsers.ParserConfigurationException;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.util.EntityUtils;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import org.xml.sax.InputSource;
import org.xml.sax.SAXException;

import android.util.Log;

public class TrovuXmlParser {
	// constructor
		public TrovuXmlParser() {
		}

		/**
		 * Getting XML from URL making HTTP request
		 * @param url string
		 * */
		
		public String getXmlFromUrl(String url) {
			String xml = null;
			try {
				// defaultHttpClient
				DefaultHttpClient httpClient = new DefaultHttpClient();
				HttpPost httpPost = new HttpPost(url);

				HttpResponse httpResponse = httpClient.execute(httpPost);
				HttpEntity httpEntity = httpResponse.getEntity();
				xml = EntityUtils.toString(httpEntity);

			} catch (UnsupportedEncodingException e) {
				e.printStackTrace();
			} catch (ClientProtocolException e) {
				e.printStackTrace();
			} catch (IOException e) {
				e.printStackTrace();
			}
			// return XML
			return xml;
		}
		
		/**
		 * Getting XML DOM element
		 * @param XML string
		 * */
		public Document getDomElement(String xml){
			Document doc = null;
			DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
			dbf.setCoalescing(true);
			try {

				DocumentBuilder db = dbf.newDocumentBuilder();

				InputSource is = new InputSource();
			        is.setCharacterStream(new StringReader(xml));    
			        doc = db.parse(is); 

				} catch (ParserConfigurationException e) {
					Log.e("Error: ", e.getMessage());
					return null;
				} catch (SAXException e) {
					Log.e("Error: ", e.getMessage());
		            return null;
				} catch (IOException e) {
					Log.e("Error: ", e.getMessage());
					return null;
				}

		        return doc;
		}
		
		/** Getting node value
		  * @param elem element
		  */
		 public final String getElementValue( Node elem ) {
		     Node child;
		     if( elem != null){
		         if (elem.hasChildNodes()){
		             for( child = elem.getFirstChild(); child != null; child = child.getNextSibling() ){
		                 if( child.getNodeType() == Node.CDATA_SECTION_NODE ){
		                     return child.getNodeValue();
		                 }
		             }
		         }
		     }
		     return "";
		 }
		 
		 
		 /**
		  * Getting img url from Description tag.
		  * @param item
		  * @param str
		  * @return
		  */
		 public String getImg(Element item, String str){
				NodeList n = item.getElementsByTagName(str);
				String ses = this.getElementValue(n.item(0));
				String imgUrl = TrovuRegEx.getMatch("<\\s*img\\s*[^>]+src\\s*=\\s*(['\"]?)(.*?)\\1", ses, 2);
				return imgUrl;
		 }
		 
		 /**
		  * Getting node value
		  * Deletes tag symbols appearing in the 
		  * @param Element node
		  * @param key string
		  * */
		 public String getValue(Element item, String str) {		
				NodeList n = item.getElementsByTagName(str);
				String ses = this.getElementValue(n.item(0));
				//return ses;  
		        String mim =ses.replaceAll("(?s)\\<.*?>", ".");
		        String fmim = mim.replaceAll("  |  ", "\n");
		        //ses.replaceFirst(regularExpression, replacement)
				//return Promjena(ses);  
				return fmim;
		}
}

Here there are the methods I use to parse the diferent kind of information. The getValue is the method used to get the "id, item, liink, title and description". The "img" tag is parsed with an other method and works fine in this Amazon Rss parse.

The idea is once I parsed the XML from the URL then in AmazonXmlParsingActivity I put inside a map the values using getValue.

If you need something more, please tell it to me

I'm sorry for my english, maybe I do a lot of mistakes for my low level :')

[Steve Luke]

Your getElementValue() method is wrong. It assumes the text in all your elements is a CDATA node, but for the title and link it is not - it is a text node.