• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Ron McLeod
  • Paul Clapham
  • Bear Bibeault
  • Junilu Lacar
Sheriffs:
  • Jeanne Boyarsky
  • Tim Cooke
  • Henry Wong
Saloon Keepers:
  • Tim Moores
  • Stephan van Hulst
  • Tim Holloway
  • salvin francis
  • Frits Walraven
Bartenders:
  • Scott Selikoff
  • Piet Souris
  • Carey Brown

Divided We Stand???

 
Leverager of our synergies
Posts: 10065
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Now something I realy like!

"After a recent on-line discussion about politics and political books in America, I wondered, what do our book buying patterns reveal about us?
In the diagram above, two books are linked if they were bought together at a major retailer on the web. I call these 'buddy books'.
<...> The actual political affiliation of each book buyer is not known, not even by the web retailer. Yet, after thousands of data points, this emergent pattern is curious.
If the network of books is a proxy for the network of individuals who read them -- what does the network map reveal? At least amongst book readers there appears to be two obvious divisions in political thought. All though academics and political pundits may read books from both clusters, the common reader obviously does not. They choose books with similar opinions again and again."
http://www.orgnet.com/leftright.html
(the article has a bigger picture and a link to another interesting paper.)
[ April 16, 2003: Message edited by: Mapraputa Is ]
 
Ranch Hand
Posts: 1340
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Interesting! Shocking?
I don't know how valid the methods are but as it stands it seems to show just how much more we're into own-world-view-affirmation than into questioning our values and considering new ways of thinking.
Once we form our own opinions about right and wrong and how the world works, can we really ever be convinced by another person's arguements? Hmmm.
Actually I haven't bought a book in about 2 yrs apart from something about EJBs which was just dull. And anyway I was always taught that reading's for girls and sissys.
 
Mapraputa Is
Leverager of our synergies
Posts: 10065
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
And anyway I was always taught that reading's for girls and sissys
You mean Real Men watch TV?
 
Richard Hawkes
Ranch Hand
Posts: 1340
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Yeah, and chicks dig scars!
 
Ranch Hand
Posts: 925
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Someone should come up with a network of .net / java books.
I wonder which books would provide the linkage between the two?
How to Be Rich - J. Paul Getty
?
 
Sheriff
Posts: 6450
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
This all seems somehow orthogonal to collaborative filtering and other similar methods used in recommender systems. Basically collaborative filtering returns tries to make assumptions about what a querant/user likes and returns results appropriately. These guesses are based on what a user's "nearest neighbors" have as likes and dislikes. To make it a bit simplistic, if there is a cluster of people who like books A, B, C, and D, and dislike book E, and you like book A, C, and D, while disliking E, the system will assume that you will like book B.
Collaborative filtering is only one of many techniques used in recommender systems. The biggest player in this area is a company called Net Peceptions, whose early work at least was based on the Group Lens research project out of the University of Minnesota. Amazon.com is a really great example of a recommender system (and I'm fairly certain they use Net Perceptions). To me this stuff goes firmly in the "very cool" column.
 
mister krabs
Posts: 13974
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Notice what he says, "the actual political affiliation of each book buyer is not known". So maybe we are looking at reserachers grabbing a bunch of books on a particular topic to do reseraach. Maybe all the right-wing books were bought by left-wing pundits who don't want to be seen buying them in their local B&N. Maybe left wingers buy more book but that doesn't mean they actually read them. I think the study proves pretty much nothing.
 
Mapraputa Is
Leverager of our synergies
Posts: 10065
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
JM: Collaborative filtering is only one of many techniques used in recommender systems. The biggest player in this area is a company called Net Peceptions, whose early work at least was based on the Group Lens research project out of the University of Minnesota. Amazon.com is a really great example of a recommender system (and I'm fairly certain they use Net Perceptions).

Thanks for the links, this Group Lens project looks particularly interesting. I wonder though if all these "recommender systems" reinforce the tendency to like what we already like. Tell Amazon once what kind of books you like, and then you will see this type of books forever on the first page (!).
TP: Notice what he says, "the actual political affiliation of each book buyer is not known". So maybe we are looking at reserachers grabbing a bunch of books on a particular topic to do reseraach. Maybe all the right-wing books were bought by left-wing pundits who don't want to be seen buying them in their local B&N. Maybe left wingers buy more book but that doesn't mean they actually read them. I think the study proves pretty much nothing.
The guy said in another paper that he analyzed Amazon's "Customers who bought this book also bought these books" statistics, so I suppose the numbers are big enough. "... we are looking at reserachers grabbing a bunch of books on a particular topic to do research" is an interesting hypothesis. What percent of USA population do you think is doing research on contemporary politics? What kind of research can one make based on "Stupid White Man" book anyway? :roll: What percent of people tend to spend money on books they never read? Etc. All modes of behavior you listed look pretty abnormal to me, in statistical sense, of course.
 
Jason Menard
Sheriff
Posts: 6450
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Originally posted by Mapraputa Is:
Thanks for the links, this Group Lens project looks particularly interesting. I wonder though if all these "recommender systems" reinforce the tendency to like what we already like. Tell Amazon once what kind of books you like, and then you will see this type of books forever on the first page (!).


Since the point of these systems is to recommend something you would be interested in, it is quite likely that you will generally see a certain type of item more frequently recommended. To make sure that it doesn't go overboard, and assume you like something that you really don't just because you once bought a gift for someone, some recommender systems allow you to ignore specific items in your profile.
Tivo also uses a recommender system, which I believe is based on collaborative filtering. Just hope you don't wake up one day to discover that your Tivo thinks you are a pregnant gay Nazi.
 
Mapraputa Is
Leverager of our synergies
Posts: 10065
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
1.
... But all of them live with a machine that seems intent on giving them such labels. It's their TiVo, the digital videorecorder that records some programs it just assumes its owner will like, based on shows the viewer has chosen to record. A phone call the machine makes to TiVo, Inc., in San Jose, Calif., once a day provides key information. As these men learned, when TiVo thinks it has you pegged, there's just one way to change its "mind": outfox it.
Mr. Iwanyk, 32 years old, first suspected that his TiVo thought he was gay, since it inexplicably kept recording programs with gay themes. A film studio executive in Los Angeles and the self-described "straightest guy on earth," he tried to tame TiVo's gay fixation by recording war movies and other "guy stuff."
"The problem was, I overcompensated," he says. "It started giving me documentaries on Joseph Goebbels and Adolf Eichmann. It stopped thinking I was gay and decided I was a crazy guy reminiscing about the Third Reich."
2. Mr. Karlsson, 26, says he "pre-emptively" found all the religious shows in his TV listings and used the "thumbs down" button on his remote control to tell TiVo he has no interest in them. (Giving three thumbs down is the best way to block a program.) After that, his TiVo recorded movies about creepy homicides. "They all have titles like 'Murder on Skeleton Isle,' " says the computer system administrator in Cambridge, Mass.
He uses the "thumbs" button to tell TiVo he hates such films. He also orders cooking shows, which softens TiVo's view of him. "I don't want it thinking I'm an ax murderer," he says.
3. Mr. Cohen, 30, has a TiVo that mysteriously assumed he wanted Korean news programs. The Philadelphia lawyer gave thumbs down to anything Korean, and his TiVo got the message. Sort of. "The next day, it recorded the Chinese news," he says.
 
Mapraputa Is
Leverager of our synergies
Posts: 10065
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I started to pay attention, and found that The Java Developer's Guide to Eclipse lists 1 DVD in "Explore Similar Items" section. I wondered what DVD is similar to "The Java Developer's Guide to Eclipse" and it's The Lord of the Rings - The Fellowship of the Ring
Second thought: how does Amazon decide which "items" are similar to the book that is not published yet?
[ April 16, 2003: Message edited by: Mapraputa Is ]
 
Mapraputa Is
Leverager of our synergies
Posts: 10065
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Ok, Swing, Second Edition is already published, so there must be better correlation.
3 "similar" DVDs:
The Day the Earth Stood Still
The Sopranos
Buffy the Vampire Slayer
Hm. Let's see now who gave this book 10 horseshoes...
 
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
how does Amazon decide which "items" are similar to the book that is not published yet?
An item can be listed in Amazon well before it's published. This allows people to find the item with searches, and you can even pre-order the item. Once you do that, Amazon can link your other purchases to the pre-ordered item.
In fact, it's kind of funny - you can sometimes infer how popular a given item is, just by looking at the other purchases linked to that item. If you see several things that have no apparent connection to the original item, that probably means that you're looking at data that came from a very small number of different users, and Amazon just listed whatever else those folks had purchased. But when there are a lot of people purchasing an item, Amazon can filter by frequency of occurrance. Only stuff that appears on several users' lists will get a link. So, the more relevant the recommended links seem to be, the more likely it is that those recommendations were derived from a large sample set, rather than one or two crackpots.
Jason - thanks for the WSJ article link, that was great. Don't you wish you were there in the crowd when "Slave Girls From Beyond Infinity" popped up among Jeff Bezos' recommendations?
 
Mapraputa Is
Leverager of our synergies
Posts: 10065
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Of course, "Buffy the Vampire Slayer" is now on my "Recently viewed items" list. I wonder how this will affect Amazon's recommendations...
 
Jim Yingst
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Dunno about Amazon, but I recommend it. Well, maybe not for you specifically, but for the rest of the world...
 
Jason Menard
Sheriff
Posts: 6450
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The cool thing is that recommendations can be inferred even with dissimilar items. What makes Amazon so good for this is that they have a large user base, and therefore a larger and more defined set of "nearest neighbors" for each user profile, as well as a diverse selection of products.
So given my earlier examples where you correlate to a group of nearest neighbors who all enjoy books A, B, C, and D, while disliking book E... Let's say that your nearest neighbors also enjoy a movie we'll call X. The system will likely infer that you will also enjoy X because other people whose profiles are similar to yours also enjoy X. This will occur even though you may have never bought a movie or browsed for a movie.
At one point my Tivo decided I was six. In a fit of nostalgia I once recorded some old cartoon off of Cartoon Network that I used to watch as a kid. I hadn't had my Tivo very long so my profile wasn't built up all that much yet, and it started recording all these kid's shows for me! I quickly went to work with the "Thumbs Down" button though and it never happened again. I'll have to try that one about proactively marking all religious shows with three thumbs down just to see what it does to my profile.
 
Jason Menard
Sheriff
Posts: 6450
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thinking about it Map, those DVD recommendations probably correlate pretty well to people who purchase the Swing book. Think about it. I don't think I would be going too far off the deep end to surmise that a good portion of the people who buy programming books also buy scifi movies as well.
 
Mapraputa Is
Leverager of our synergies
Posts: 10065
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Ha! All great minds think about the same problems. Totally accidentally I clicked on Andy Oram's old link and found an article I read a while ago, and even wanted to quote sometimes.


"... There are many other reasons popularity ranking can be skewed, too: someone who happens to be first to raise a topic is likely to be noticed just for that accomplishment; someone who addresses a narrow technical audience may not get adequate recognition for his or her contributions; someone who's a lone nut may be referenced a lot just because no one else expresses the same ideas, etc.
So collaborative filtering appeals to me even more than popularity ranking. A collaborative filtering system lets you rate things (movies, books, politicians--anything) and compares your ratings to other people's ratings. If someone has movie preferences similar to yours, you are likely to agree on the next movie to come along.


This was the part I liked most:


When I want to educate myself regarding a topic, my first step is to find a place where interested people congregate (it could be a mailing list, if I am doing my research in virtual mode) or a collection of useful documents. When I find people who impress me with their insights or who simply intrigue me with their points of view, I spend more time reading what they have to say and ask them for pointers to new material.
This technique uses affinity between individuals, as collaborative filtering does, but the individuals are actively seeking affinity rather than passively waiting for it to emerge from a collaborative filtering system.
Most subtle, perhaps, is the way I discover new topics of importance by following what interests the people I respect. For instance, if I learn from someone's views in a particular software area and find that he's becoming obsessed over some piece of hardware, I decide that it's time to look into that hardware. I don't simply screen out this new information because it's different from the software area that we've always talked about.


I guess this "passive" mode is precisely what bothers me in Amazon's approach. I do not want to know what people like me like (no pun intended), I want to know what people who are better than me like. It's A. Maslow's meme, I am sure, that contaminated my mind. He said in one of his books that some people have ability to make the best choices, what is good and healthy for human beings. If other people followed their choice, they would only benefit. Something like that, I forgot details. I employ this strategy since then, and it works fine. Find somebody who is a deep thinker in software field, and chances are his favorite books or movies are also damn interesting.
[ April 18, 2003: Message edited by: Mapraputa Is ]
 
Jason Menard
Sheriff
Posts: 6450
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Originally posted by Mapraputa Is:
I guess this "passive" mode is precisely what bothers me in Amazon's approach. I do not want to know what people like me like (no pun intended), I want to know what people who are better than me like.


Amazon's implementation does actually support this to some extent. User's are allowed to create what Amazon calls Listmania Lists and So you'd like to.. Guides which give information regarding the likes of the person who creates one. So if you are able to find a user whose insight you value and they have created one of these things, then I think this does what you want.
I've done some academic research into this stuff and I am really impressed by Amazon's implementations of various recommender technologies. If you are interested in playing with their system, my advice would be to check out the Recommendatios, Friends & Favorites, and Notification Systems areas, which all include various recommender techniques. I would also recommend reading this paper (don't miss the appendices) from the GroupLens Project if you want to have some context of understanding for the various recommender methods that Amazon and others employ. If you are interested specifically in collaborative filtering, GroupLens: Applying Collaborative Filtering to Usenet News is a classic paper on the subject.

Find somebody who is a deep thinker in software field, and chances are his favorite books or movies are also damn interesting.


Not necessarily imho, not unless they are otherwise "like" you. What I mean is that Stephen Hawking for instance could really have had a deep appreciation for Rodgers and Hammerstein musicals and the like. I hate these! Just because somebody is a genius doesn't mean they have tastes that will coincide with your own. But I know what you are saying and I do agree that there is some value in knowing what these people "like".
[ April 18, 2003: Message edited by: Jason Menard ]
 
Thomas Paul
mister krabs
Posts: 13974
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I find that most of the recommendations that amazon makes for me are things that I would never buy. Sci-fi is a big one. It seems like every geek programmer must love sci-fi and I have no interest in it. My daughter and I prefer detective movies from the 30's and 40's. Tonight we are watching two "Thin Man" movies.
 
You ought to ventilate your mind and let the cobwebs out of it. Use this cup to catch the tiny ads:
Thread Boost feature
https://coderanch.com/t/674455/Thread-Boost-feature
reply
    Bookmark Topic Watch Topic
  • New Topic