A Tech Lover’s Luddian Emotions

I have been conflicted for a long time with technology, and hype. Technology is wonderful in that it gives us productivity or efficiency, but what are do we trade for them? It isn’t always free, we still have to put in something, and maybe we don’t even see all the resources that are devoted to the process.

I have a tendency to feel Luddian about things, but it comes from knowledge not ignorance. Most things have merits, that, I can’t deny, but it also has somewhat hidden ramifications for it’s use. Cellphones. RSS. Search Engines. Social Networks. All have their positives, but simultaneously negatives.

We have reached a point of decentralization, so that we can do almost anything, anywhere; we’ve also managed to place a shackle upon ourselves, that is virtually always there, (or always virtually there.) We have created a world that we can’t walk away from, because it is so easy to connect and get drawn in. We can check the news, our portfolios, our relationships with family and friends, using the technology at our fingertips via text, photos, video, audio all on a device that can be kept in our pocket. At this point we have more knowledge that is more ubiquitous than at any other time, but with it we’ve become forgetful.

I’m stupid. I can’t give you a single friends phone number off the top of my head. My grammar and spelling is often atrocious, without spellcheck. I often have to check definitions because I only have half a clue of what a word might mean. I can find information, but it’s often not of the quality that is necessary, for what I want.

We gave up our freedom, of being unavailable for evenings, weekends, weeks, in order to always have access to the outside world, with cellphones. Then we gave up our freedom to not be bombarded by news, and information. How much of the news you see every day, has an immediate or profound effect on you? Noise is growing faster than the valuable pieces, and search is succumbing to it.

Last night, I spent 2 hours searching for an article I had read about the creation of Zork. Unfortunately, the article was mainly on the MDL language, with Zork  only as an example. Searching for it, Google gave me nothing but junk. Stumbleupon and Delicious  gave me nothing. I only lucked out in that I had shared the article with Friendfeed, and Friendfeed’s search was actually functional. It shouldn’t take that long to find what you’re looking for. That was 2 hours I could have spent actually doing something, by the end I was exhausted, and didn’t even really want to do what I had started the search for.

Everything comes with the sacrifice, but it seems that I’m having to sacrifice more and more to gather the value from all the systems that I use. I broke away from my cell phone last year, it was great for setting up the occasional meeting with friends, and in cases of emergency, but over all it was just a nuisance. I like to escape from constant distraction and information, because it’s what I deal with on a day to day basis. Somewhere I feel that we lost our way, but at the same time I’m damn amazed at what is possible now. I stand torn, in what I know, and my opinions on what I know.

This was written on January 21st, 2011.

Chronological Data’s Influence On Relevancy Analysis

Last night , I read Content should be experienced by relevance and importance and interestingness, not chronologically,  after Louis Gray shared it on Friendfeed. This is my take on chronological and other relevancy metrics. While, I agree with Garry on the outcome; I don’t  want to shrug off chronology as lacking importance, is a great tool for weighting an objects relevance.

Chronological data is a very nice weight to have when you’re looking at the whole set of objects. Chronology isn’t as important for idea’s or thoughts, as it is for news, but when looking at the subset of most actual news, it is much better.  Cadmus succeeds because it focuses on a small chronological window, or at least it does in my case, showing me the most relevant items in the last 24 hours, as well as a focused source of input, Twitter. Twitter, and most social tools, have high-entropy in relevance over a sustained period, so that if you really want to provide relevant or important information, focusing on what has happened in the past 24 hours is a great idea. Unfortunately, the web isn’t just Twitter, Facebook, or other tools, there are thousands of blogs, and news sources that are also relevant.

I’m going to use the example of a feed reader, considering that’s where I was focused when looking at these things, and thus have more insight into the discussion at hand, in that area. Trying to determine relevance on items, particularly news, doesn’t work so well when you’re focusing on a set of items you’ haven’t read, over a period of weeks, months, years, or even a few days, and seeking relevance/importance. What you end up with is possibly having sets of news that is outdated, being more relevant than current news, or an item that doesn’t fit the users interests.

Some real quick techniques, to boost relevance involve chronological data, though it’s not as necessary in the later stages of relevancy, it plays a huge role in cutting the set down to size for analysis. Here are a few methods of using chronological data for quickly sculpting more relevant information.

  • Create a window, static or sliding, this helps capture and condense echo. (48-72 hours is good)
  • Over a period, larger than your initial window, you can remove stale items, by comparing condense sets, that are on the topics that are more current.

Chronology is an extremely quick and dirty tool, but it can help tremendously, in narrowing the items down quickly. So that the data that needs to be processed for each user is much smaller, however it is far from the be all end all of the process for determining relevant data. The list of other items for determining importance or relevancy:

  • An external source weighting similar to PageRank — allowing high-value content to be controlled by peers as well as sharing it’s clout. (Source-Data relationships)
  • A personalized weighting based on your relationships similar to EdgeRank — allowing your personal interactions to show trust and interest in items. (Human-Human relationships)
  • A personalized weighting, based on your habits, and usage of various items similar to APML — allowing your content usage to be analyzed and weighted. (Human-Data relationships)
  • An aggregate weighting of both Edgerank & APML, to determine, weighting of topics based on human relationships and habit comparison, an idea like GAP(ML) — allowing your common interests and friendship to expose a more complex set of relevant data. (Human-Data-Human relationships)
  • A set of common related data carriers, and user relationship with his sources, somewhere in between Edgerank and Pagerank — allowing quick analysis and overview of sources, to determine what is important currently, as well as what is important among the sites the user trusts. (Source-Source & Source-Human relationships)

Those are the main relationships, in my mind, though there are a few others, such as media relationships(text, audio, video) not all people prefer the same type of information format. Tools that the user is using, you may want to provide data to the user in a different manner depending on how and what they are using to observe the data, or where they observe the data. If the user is the key, then the relationships and objects around the user are most definitely the teeth, and you have to hit as many tumblers as you can, without getting stuck.

These are just the ones I’ve focused on, and I’m sure their are others just as value that I have skimmed over, but this should provide a good, base for starting out, and there are probably a million little tweaks and touches that I skipped.  To quote Garry, because he was right.

The field is still a bit wide open because few people have both the dataset to work and test on, AND the financial backing to see the project all the way through.

We will definitely push the boundaries over the next few years, and we’ll have a better order for our information, I have no doubt in this. However, I’m betting many individuals will still rely on the very simplicity that we rely today: Chronological ordering. And even if it isn’t shown that way on the surface, deep down, it will be at the very base of relevance. Some under-the-radar companies, in this area, are Bit.ly and BagtheWeb — bundles are an excellent source of the information required for relevancy*, Quora and Stack Overflow — Q&A is a huge resource for the personal interests, Cadmus, Hunch, and My6Sense — all have experience working with this, and I have no doubt all three will only get better.

*- I may be biased here, I’ve been working on and off with bundling, for close to 2 years, but ultimately got lost in perfecting how the data was stored.

Adobe Pops Above Google to Flash Apple (and Opera) Users

Jesse Stay, brought to my attention earlier, with the screenshot below the code, that Adobe was activating pop-ups through Google Adsense; of course this shouldn’t happen, since Google doesn’t allow such actions, in their ads. I went to check in out for myself, and got nil. I immediately assumed that it was limited to Mac, and went in search of the User-Agent check, and found it after about 5 minutes.

Update: The rest of this article is a bit technical, if you would like a less technical description, you should go read Adobe and Google Sitting in a Tree.

Below is the source of the issue:


document.write('<!-- Template Id = 2,593 Template Name = Banner Creative (Flash) - In Page --><!-- Copyright 2006 DoubleClick Inc., All rights reserved. --><script src=\"http://s0.2mdn.net/879366/flashwrite_1_2.js\"><\/script>');document.write('\n');

function DCFlash(id,pVM){
var swf = "http://s0.2mdn.net/1295336/Adobe_Flash_WeLoveTechStandAlone_300x250_std.swf";
var gif = "http://s0.2mdn.net/1295336/Adobe_Flash_WeLoveTechTandem_300x250_img.gif";
var minV = 8;
var FWH = ' width="300" height="250" ';
var url = escape("http://ad.doubleclick.net/click%3Bh%3Dv8/399a/f/16c/%2a/z%3B224918296%3B0-0%3B0%3B48697163%3B4307-300/250%3B36759992/36777870/1%3B%3B%7Esscs%3D%3fhttp://adclick.g.doubleclick.net/aclk?sa=l&ai=B7rXGceHsS9PODOThlQfNrvjSDe_sysIBAAAAEAEgr86-CjgAWM_pofIUYMnO7Y2ApfgRsgEOdGVjaGNydW5jaC5jb23IAQnaATRodHRwOi8vdGVjaGNydW5jaC5jb20vMjAxMC8wNS8xMy9jaHJvbWUtb3MtcHJvZ3Jlc3MvmAJkwAIC4AIA6gISVGVjaGNydW5jaF8zMDB4MjUw-AL00R6QA-ADmAOsAqgDAeAEAQ&num=0&sig=AGiWqtxB0NIJCJR5KJ5OngVkuvd_Qw20Dw&client=ca-pub-6181816114362650&adurl=http%3a%2f%2fwww.adobe.com/choice%3Fsdid%3DGXRVD");
var wmode = "opaque";
var bg = "same as SWF";
var dcallowscriptaccess = "never";

var openWindow = "false";
var winW = 600;
var winH = 400;
var winL = 0;
var winT = 0;

if(typeof(encodeURIComponent)=="function"){url=encodeURIComponent(unescape(url));}
var fv='"clickTag='+url+'&clickTAG='+url+'&clicktag='+url+'"';
var bgo=(bg=="same as SWF")?"":'<param name="bgcolor" value="#'+bg+'">';
var bge=(bg=="same as SWF")?"":' bgcolor="#'+bg+'"';

function FSWin(){
 if((openWindow=="false")&&(id=="DCF0"))alert('openWindow is wrong.');
 if((openWindow=="center")&&window.screen)
 {winL=Math.floor((screen.availWidth-winW)/2);winT=Math.floor((screen.availHeight-winH)/2);}
 window.open(unescape(url),id,"width="+winW+",height="+winH+",top="+winT+",left="+winL+",status=no,toolbar=no,menubar=no,location=no");}this.FSWin = FSWin;

ua=navigator.userAgent;
if(minV<=pVM&&(openWindow=="false"||(ua.indexOf("Mac")<0&&ua.indexOf("Opera")<0))){
 var adcode='<object classid="clsid:d27cdb6e-ae6d-11cf-96b8-444553540000" id="'+id+'"'+FWH+'>'+
 '<param name="movie" value="'+swf+'"><param name="flashvars" value='+fv+'><param name="quality" value="high"><param name="wmode" value="'+wmode+'"><param name="base" value="'+swf.substring(0,swf.lastIndexOf("/"))+'"><PARAM NAME="AllowScriptAccess" VALUE="'+dcallowscriptaccess+'">'+bgo+
 '<embed src="'+swf+'" flashvars='+fv+bge+FWH+' type="application/x-shockwave-flash" quality="high" swliveconnect="true" wmode="'+wmode+'" name="'+id+'" base="'+swf.substring(0,swf.lastIndexOf("/"))+'" AllowScriptAccess="'+dcallowscriptaccess+'"></embed></object>';
 if(('j'!="j")&&(typeof dclkFlashWrite!="undefined")){dclkFlashWrite(adcode);}else{document.write(adcode);}
}else{
 document.write('<a target="_blank" href="'+unescape(url)+'"><img src="'+gif+'"'+FWH+'border="0" alt="" galleryimg="no"></a>');
}}

var pVM=0;
var DCid=(isNaN("224918296"))?"DCF0":"DCF224918296";
if(navigator.plugins && navigator.mimeTypes.length){
 var x=navigator.plugins["Shockwave Flash"];if(x && x.description){var pVF=x.description;var y=pVF.indexOf("Flash ")+6;pVM=pVF.substring(y,pVF.indexOf(".",y));}}
else if (window.ActiveXObject && window.execScript){
 window.execScript('on error resume next\npVM=2\ndo\npVM=pVM+1\nset swControl = CreateObject("ShockwaveFlash.ShockwaveFlash."&pVM)\nloop while Err = 0\nOn Error Resume Next\npVM=pVM-1\nSub '+DCid+'_FSCommand(ByVal command, ByVal args)\nCall '+DCid+'_DoFSCommand(command, args)\nEnd Sub\n',"VBScript");}
eval("function "+DCid+"_DoFSCommand(c,a){if(c=='openWindow')o"+DCid+".FSWin();}o"+DCid+"=new DCFlash('"+DCid+"',pVM);");
//-->

document.write('\n<noscript><a target=\"_blank\" href=\"http://ad.doubleclick.net/click%3Bh%3Dv8/399a/f/16c/%2a/z%3B224918296%3B0-0%3B0%3B48697163%3B4307-300/250%3B36759992/36777870/1%3B%3B%7Esscs%3D%3fhttp://adclick.g.doubleclick.net/aclk?sa=l&ai=B7rXGceHsS9PODOThlQfNrvjSDe_sysIBAAAAEAEgr86-CjgAWM_pofIUYMnO7Y2ApfgRsgEOdGVjaGNydW5jaC5jb23IAQnaATRodHRwOi8vdGVjaGNydW5jaC5jb20vMjAxMC8wNS8xMy9jaHJvbWUtb3MtcHJvZ3Jlc3MvmAJkwAIC4AIA6gISVGVjaGNydW5jaF8zMDB4MjUw-AL00R6QA-ADmAOsAqgDAeAEAQ&num=0&sig=AGiWqtxB0NIJCJR5KJ5OngVkuvd_Qw20Dw&client=ca-pub-6181816114362650&adurl=http%3a%2f%2fwww.adobe.com/choice%3Fsdid%3DGXRVD\"><img src=\"http://s0.2mdn.net/1295336/Adobe_Flash_WeLoveTechTandem_300x250_img.gif\" width=\"300\" height=\"250\" border=\"0\" alt=\"\" galleryimg=\"no\"></a></noscript>\n');
</script>

<script src="http://s0.2mdn.net/879366/flashwrite_1_2.js">
function dclkToObject(id) {
 if(document.layers){
 return (document.layers[id])?eval(document.layers[id]):null;
 }
 else if(document.all && !document.getElementById){
 return (eval("window."+id))?eval("window."+id):null;
 }
 else if(document.getElementById && document.body.style) {
 return (document.getElementById(id))?eval(document.getElementById(id)):null;
 }
 }

function dclkFlashWrite(string){
 document.write(string);
}

function dclkFlashInnerHTML(htmlElementId,code){
 var x=dclkToObject(htmlElementId);
 if(x){
 if(document.getElementById||document.all){
 x.innerHTML='';
 x.innerHTML=code;
 }
 else if(document.layers){
 x.document.open();
 x.document.write(code);
 x.document.close();
 }
 }
}
</script>
Adobe Ad

Adobe pops up on a Mac

The interesting thing I spotted quite quickly is that it is also going after Opera, don’t ask me why, that’s pretty obscure. I immediately tested it out, just to see, and ended up getting the pop-up.

After looking at the code for about a half hour, I still don’t know what everything is, and exactly how it’s getting past Google, I also don’t know how ad’s are created, because I’ve never bothered to look at it.  So I’m not sure, if this is something anyone could execute, or if Google is allowing it. So it’s possible that there is a vulnerability in Adsense.

What I can tell after looking at the code, is that they are targeting Apple, and Opera, users, as well as using javascript to activate flash, in the background.

First they are setting the DCid as either DCF0 or DCF224198296, this should always validate as false and the will set DCid as DCF224198296. Then, it goes on to check if the browser uses plugins and has at least any values, if it results in true it attempts to setup ShockwaveFlash and perform a version check on it, setting the variable pVM to the version number. If that statement failed, it assumes you are using IE and initializes using ActiveX.

I don’t understand exactly what is going on in the eval, so I can’t say much about it, besides it calls DCFlash with the DCid(“DCF224198296″) and the pVM(“Flash Version”).

I apologize if the rest of this is rushed for now, I’m getting a bit tired.

The DCFlash function then initializes an assortment of variables, before setting up the FSWin function.
FSWin checks to see if the window is already open or the DCid was set in error to DCF0, it then checks to see if the window is centered and gather your screen size for offsets on the window border. Following FSWin is the window initialization, which disables all navigation, in that window.

Next,  is the User-Agent analysis, which first check to make sure that your current Flash version is at minimum Vers. 8, it then checks to see if you are using either a Mac or Opera, if you aren’t the value is less than 0, returning true, this then compares in with window==”false”, which is true, in an or statement, which will return false, if both are set. If this test of the browser conditions fail, either using an old version of Flash, or using a Mac, or Opera it will default to just the default hyperlinked gif. Otherwise, it sets up Flash to be displayed in the window pain.

During the course, of writing this the test ad I was using has disappeared, but I’ll see if I can gather any more of the code. One thing I found odd, is that the ad was stored within an iFrame, which I couldn’t find with any of Google’s other Adsense ads, I managed to find a representation of using an iFrame, after I woke up.

The Influence of Homogeneity on Choice and the Web

One thing that consistently occurs in markets is a trend towards homogeneity, it’s nothing new, it’s been happening for millennia. The occurrence in the web is a bit disturbing, however, because of time and focus. Alas, I’m getting ahead of myself, I should probably talk about the various forms of homogeneity, that occur naturally and in our daily lives.

Homogeneity, in nature, it is as common as a step in the evolutionary process. As species adapt, the mean difference, across the whole corpus of the species, becomes smaller, and the species pick up both positives and negatives, that affect the species future. The huge negative for homogeneity is that the corpus, minus the mutated(outliers), is that susceptibility to a common tragedy. A tragedy, by contagion,  can be common, if a disease does infect one entity, it can easily spread to those that are also susceptible, though the entities with mutated genetics, assuming theses genetics, increase resistance or provide immunity to the disease, or any common tragedy.  If the tragedy is large enough, it can cause the mean difference to go up, or even hit an maxima,  which it comes back down from in favor of the  mutates.

Homogeneity, in business and our daily lives, occurs quite frequently, and it is an influencing factor over choice. To look into the idea of choice and homogeneity, you don’t have to look further than skin deep, or in this case clothing. Every one of us has a desire to fit in, and this desire results in us commonly grouping, even, subconsciously with those who are like us. From this desire, we find ways to look, sound, or interact the same way; our desire also has a side effect, it removes the necessity of thought in many circumstances, just go with what everyone else is doing. Of course, there will always be a few who don’t want to be classified, or collated with others, sadly they create their own group, through these actions: non-conformists.

To see the corporate side of homogeneity, look into the restaurant/fast food industry, or supermarkets. Mom & Pop’s have been gobbled up by the McDonalds and Wal-Marts , because the layperson doesn’t know what they have to offer. If you go with one of these household names, you have a good idea that the food is going to be decent, or that the store will have what you’re looking for within its doors. This homogeneity, decreases local competition, but it’s okay, because it saved the average consumer time and money, because these chains get reductions for ordering extremely large amounts of goods. Of course, some people will stick to the Mom & Pop’s, to be contrarian, or because they know it just as well and it has become ritual.

When it comes to the web, however, the steps toward homogeneity become much easier, but there is even fewer checks and balances, than in any of the other cases. We interact with the web on a time basis, and this time is limited, so we find a subset of sites to stay in constant contact with, normally staying within a triumvirate: search, networking, and news-history. However, the common solutions for these problems are reduced to a common set of sites, there are alternatives, but it requires more rigor on part of the consumer. So what do we do, we choose, by what provides the most tools, where are my friends, and how can I find out more.

What happens when you’re playing this zero-sum game of choice, Louis Gray says there is no zero-sum game?  You end up selecting those that might not be the best, but save you time and trouble. Want to use e-mail, read blogs, or just IM with friends, you can go use any random email host, any old RSS-reader, or link walk the sites, or anyone who offers an IM service, OR you can just use Google, and get all of these services simultaneously, plus several dozen other services.  Do you see what just happened? Multiple services where just reduced, they were hit by a common tragedy, and now there is one hyper-efficient service provider, which most people are going to use because it’s simple, and they don’t have to think about where they are going to go, or what they are going to do.  Where can I share images, discuss things with my close friends and family, and provide a set of personal information for people that people can use as entrance sources? Well there are a large number of services that will let you share images, and any number of places and ways to share that information, but to truly access everyone, without making them do work hard, and that site is becoming Facebook. The case is you don’t see a corresponding 1-1 gain loss, gains are primarily individualistic, while losses are primarily distributed, there are cases where the inverse occurs, but they are few and far between.

These companies are becoming goliaths, that are going to harm the web, if they continue to grow, it won’t happen immediately, but even now Facebook is trying to change the rules. And you can say all you want that there are other services out there, I’ll admit that, but when you’re playing a zero-sum game, based on how much time you spend interacting in different locations, you have to focus on where you’ll get the optimal return.  There will always be alternatives, for those who truly want them, but for the general public, they don’t mind as far as they know, everyone does the majority of the same things on the web. Until, something happens that causes the homogenous species, to see what the mutates have already seen and adapted for we’re looking at an interesting ride for the next 2-3 years.

Graph Attention Profiles – GAP(ML)

This was an idea I had earlier this morning about how to optimize social ad placement services, (MyLikes (Aff. Link), Magpie, etc. ) These services work by placing ads into the a social stream , I like MyLikes model, they let you decide what to put into the stream based on what you like, but this doesn’t factor in what your followers like, the ad needs to be relative to them, not you*. Thinking about how to determine the relevancy to a group, I came up with an idea based around averaging individual APMLs(Attention Profiling Mark-up Language).

I haven’t thought it out fully, it’s only been a few hours, but using APMLs as the starting ground. You sum the weights, per topic, for all of your followers and then divide by #number of followers, to get the APML for your Social Graph, per network which I’m calling GAP currently. I see this as an extended OPML format for APMLs , handling not only weights of relevant interest, but also handling access to the APMLs monitored by the graph.

One thing that would conflict with the APML format, which the GAP could stay very close to, is what is deemed Explicit Data. You aren’t the one determining relevancy, so it isn’t necessary. I’d either use or replace it for something that handles the APML list being monitored, the list becomes the explicit data for the weighting, but it also allows you to weight the APML’s individually as well, I don’t know that this is necessary, but it allows accessibility to possibly increase relevance to your graph, based on who is likely to interact more with you.

So this is just a thought, about a open-method for sharing graphs and relevance between services, rather than every service handling a proprietary model of the graph, and a proprietary model of relevant data. First things first, is that we need support for APML, which we have Chris Saad to thank for, then we can handle how we manage our networks relevancy.

One final issue with the GAP is that it has a specific use case, is that it is a way to share graphs and relevancy to exterior networks, but the file size for the GAP if it handled all the networks simultaneously it would become quite large, implicit data would be 1 line per topic, per network, and explicit data would be 1 line per person, per network. For early adopters and people with large following bases this could become quite large, even for a regular user on one network it would likely be 300-1000 lines.

*= MyLikes already uses a similar model, influenced by clicks per ad and number of ads you share. MyLikes Influence Rank