Affective Computing

One of the more interesting technologies that has been developing is called affective computing. It’s about analyzing observations of human faces, voices, eye movements and the like to understand human emotions — what pleases or displeases people or merely catches their attention.  It combines deep learning, analytics, sensors and artificial intelligence.

While interest in affective computing hasn’t been widespread, it may be nearing its moment in the limelight. One such indication is that the front page of the New York Times, a couple of days ago, featured a story about its use for television and advertising. The story was titled “For Marketers, TV Sets Are an Invaluable Pair of Eyes.”

But the companies that were featured in the Times article are not the only ones or the first ones to develop and apply affective computing. IBM published a booklet on the subject in 2001.  Before that, in 1995, the term “affective computing” was coined by Professor Rosalind Picard of MIT, who also created the affective computing group in the MIT Media Lab.

In a video, “The Future of Story Telling”, she describes what is essentially the back story to the New York Times article.  In no particular order, among other companies working with this technology today, there are Affectiva, Real Eyes, Emotient, Beyond Verbal, Sension, tACC, nVisio, CrowdEmotion, PointGraB, Eyeris, gestigon, Intel RealSense, SoftKinetic, Elliptic Labs, Microsoft’s VIBE Lab and Kairos.

Affectiva, which Professor Picard co-founded, offers an SDK that reads emotions of people at home or in the office just by using web cams.  Here’s a video that shows their commercially available product at work:

Similarly, Real Eyes also
offers a commercial product that analyzes the reactions of what people
see on their screens. Here’s their video about real-time facial coding:

two previous products have obvious application to web marketing and
content. So much so, that some predict a future in which affective
technology creates an “emotion economy”.

But affective computing
has longer term applications, most especially in robotics. As human-like
robots, especially for an aging population in Asia, begin to be sold as
personal assistants and companions, they will need to have the kind of
emotional intelligence about humans that other human beings mostly have
already. That’s likely to be where we will see some of the most
impactful uses of affective computing.

Over the last couple of
years, Japan’s Softbank has developed Pepper, which they describe as a
“social robot” since it aims to recognize human emotion and shows its
own emotions. Here’s the French software company behind Pepper  —

are others doing the same thing. At Nanyang Technological University,
Singapore, another social robot, called Nadine, is being developed.  See

these social robots and affective computing overall still needs much
development, but already you can sense the importance of this

© 2017 Norman Jacknis, All Rights Reserved


Rules Of The Road For Robots

When we drive in our cars, we mostly have a sense of common rules for the road to keep us all safe. Now that we begin to see driverless cars, there are similar issues for the behavior of those cars and even ethical questions.  For example, in June, the AAAS’s Science magazine reported on a survey of the public’s attitudes in answer to the story’s title: “When is it OK for our cars to kill us?

Driverless cars are just one instance of the gradual and continuing improvement in artificial intelligence which has led to many articles about the ethical concerns this all raises. A few days ago, the New York Times had a story on its website about “How Tech Giants Are Devising Real Ethics for Artificial Intelligence”, in which it noted that “A memorandum is being circulated among the five companies with a tentative plan to announce the new organization in the middle of September.”

Of course, this isn’t all new. About 75 years ago, the author Isaac Asimov formally introduced his famous Three Laws of Robotics:

1.     A robot may not injure a human being or, through inaction, allow a human being to come to harm.

2.    A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.

3.    A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws


Even before robots came along, ethics was focused on the interactions between people and how they should not harm and conflict with each other – “do unto others …”. As artificial intelligence becomes a factor in our world, many people feel the need to extend this discussion to robots.

These are clearly important issues to us, human beings. Not surprisingly, however, these articles and discussions have a human-centric view of the world.

Much less – indeed very little – consideration has been given to how artificial intelligence agents and robots interact with each other. And we don’t need to wait for self-aware or superhuman robots to consider this.

Even with billions of not so intelligent devices that are part of the Internet of Things, problems have arisen.


This is, after all, an environment in which the major players haven’t yet agreed on basic standards and communications protocols between devices, never mind how these devices should interact with each other beyond merely communicating.  

But they will interact somehow and they will become much more intelligent – embedded AI. Moreover, there will be too many of these devices for simple human oversight, so instead, at best, oversight will come from other machines/things, which in turn will be players in this machine-to-machine world.

The Internet Society in its report on the Internet of Things last year at least began to touch on these concerns.

Stanford University’s “One Hundred Year Study” and its recently released report “ARTIFICIAL INTELLIGENCE AND LIFE IN 2030” also draws attention to the challenges that artificial intelligence will pose, but it too could focus more on the future intelligent Internet of Things.

As the inventors and producers of these things that we are rapidly connecting, we need to consider all the ways that human interactions can go wrong and think about the similar ways machine to machine interactions can go wrong. Then, in addition to basic protocols, we need to determine the “rules of the road” for these devices.

Coming back full circle to the impact on human beings, we will be affected if the increasingly intelligent, machine-to-machine world that we depend on is embroiled in its own conflicts. As the Kenyan proverb goes (more or less):


“When elephants fight, it is the grass that suffers.”

© 2016 Norman Jacknis, All Rights Reserved


The Internet Of Things Spreads And Raises Concerns

Although it’s not growing as fast as some people thought a year ago, the Internet of Things continues to be deployed.

we often think of the sensors and other aspects of the Internet of
Things being part of the management of large cities or industrial
enterprises, some of the most interesting developments have occurred,
but gone largely unnoticed, on the farm.

A few weeks ago, two articles about this appeared – “The Dawn Of The Smart Farmer: Sowing Sensors And Connecting Crops” and a blog by Steve Lohr (one of the NY Times very best technology reporters) “The Internet of Things and the Future of Farming”.

things that are being connected have, so far, been digital devices. But
the Korean company Naran is introducing a micro-robot, called the Push.
Any non-digital switch, like a light switch, can be controlled by this
nearby device, which in turn is connected to a small Prota computer that
tells the little robot when to turn the switch on or off based on a set
of rules set up by the user. There’s also a smartphone app for
preparing these rules or simply controlling the robot directly. 


Perhaps the greatest potential
of the Internet of Things is as a step to other ways of extending the
Internet. The University of Virginia announced a new way to use regular LED lights:

like using fiber optics to communicate – only without the fiber.
Imagine connecting to the Internet through the same room lights that
brighten your day. A University of Virginia engineering professor and
her former graduate student are already there… Their breakthrough means
that data can be transmitted faster with light waves using no more
energy than is already required to run the lights.”

Of course,
problems, like security, are always an issue, even in surprising
quarters. Vint Cerf, the unofficial father of the Internet and Chief
Internet Evangelist at Google, worries that the software behind the
Internet of Things has bugs. Last week, he publicly confessed that “Sometimes I’m terrified by it”.

Andy Greenberg of WIRED magazine has been particularly active reporting on these issues. In July, he wrote how “Hackers Can Disable a Sniper Rifle—Or Change Its Target”.


In July, starting with another article in WIRED,
the hacking of cars built by major American car manufacturers made big
news. That led to a recall that was a nuisance, a necessary nuisance,
for car owners.

On a more positive note earlier this month, another reporter at WIRED followed up with an article titled “Researchers Hacked a Model S, But Tesla’s Already Released a Patch
noting that the same Internet of Things which opened up a vulnerability
could also be used by smart companies to close those doors quickly.

© 2015 Norman Jacknis, All Rights Reserved


Big Brother or Fun Uncle?

The recent news about NSA collection of data and Google’s ever expanding collection of personal information have increased concerns about privacy.  So some people are worried the use of sensors all over – the Internet of Things or Everything – will lead to Big Brother.   

A few of my recent blog posts show creative uses of the Internet everywhere.

I think of this as more like your Fun Uncle than Big Brother 🙂

Happy Fourth of July!

© 2013 Norman Jacknis


Who Needs To Pay Attention To The Internet of Things?

There’s an increasing drum beat of news about the “Internet of Things”.   There’s even an acronym for it – IoT.   

IoT refers to the increasing number of devices on the Internet that report the status and location of physical things.  This covers everything from the location of your smart phone to where a package might be to the condition of your pulse to the condition of a highway.   (That’s why it is also sometimes referred to as the Internet of Everything.)

All this data has also led to people talking about “Big Data” and the need for analytical software to make sense of it all.

Less often noted is that things connected to the Internet can communicate with each other.  We’ve only begun to think about the practical and fundamental issues this phenomenon will raise.  

On a practical level, this machine-to-machine communications needs to be managed by people not through on-off switches or gauges, but through policies that can be operated at the same speed as the machines – not at the slow speed of human awareness and decision making.

The benefits can be striking.  For example, a bridge whose sensors are detecting potential cracks in load-bearing columns can ask the street light to turn red to stop traffic and also tell the police dispatch system to get a couple of police cars out to redirect that traffic.

Of course, the complexity of a global system that connects all these devices is mind boggling.  This global system has the potential for unpredictable and perhaps disastrous behavior.

That alone should get the attention of public leaders.

Now, most of the advertising and news from technology companies has focused on how corporations can use the Internet of Things.  Surely they can.  Just think of any company that ships things and needs to know the condition of the shipped items and there locations.

Companies are usually responsible for their own office and manufacturing space.  Even including shipments or goods, any individual company has to worry about at most millions of square feet.

However, governments are uniquely responsible for what goes on in a particular territory, which can be many tens, hundreds, thousands or even millions of square miles.  Eventually, all this territory will be covered by sensors, which will greatly outnumber everything else on the Internet. 

By the way, the Internet of Things is not something way off in the future.  Today, the number of physical devices connected to the Internet is already six times the number of people on the Internet, even though there are two billion of those people.  By 2020, just a few years away, there will be 50 billion connected devices.

It’s time for government leaders to start focusing on IoT as a policy concern and as a tool for managing their infrastructure and territory.

© 2013 Norman Jacknis