All posts tagged Tech

If you’re into systems that can share data among each other in real time, this has been a good week. On Tuesday, Yahoo open sourced its version of the popular Storm stream-processing software that’s able to run inside Hadoop clusters. Then, on Thursday, Facebook detailed a system called Wormhole that informs the platform’s myriad applications when changes have occurred in another, so that each one is working from the newest data possible.

The Yahoo work is actually pretty important. Among the features Hadoop users have been demanding from the platform is a transition from batch-processing-only mode into something that can actually deal with data in real time. The reason for the demand is quite simple: Although being able to analyze or transform data minutes to hours after it’s generated is helpful for certain analytic tasks, it’s not too helpful if you want an application to be able to act on data as it hits the system.

A service like Twitter is a prime example of where Storm can be valuable. Twitter uses Storm to handle tweets so users’ Timelines are up to date and do things like real-time analytics and spotting emerging trends. In fact, it was Twitter that open sourced Storm in 2011 after buying Storm creator Backtype in order to get access to the technology and its developers.

Among web companies, Storm has become quite popular as a stream-processing complement to Hadoop since then. And now Yahoo has made possible a much tighter integration between the two — even to the point that Storm can borrow cycles from batch-processing nodes if it needs some extra juice. That’s a valuable feature — just last week I heard Twitter engineer Krishna Gade bemoan Storm’s auto-scaling limitations during a talk at Facebook’s Analytics @ Web Scale event.

Krishna Gade talking Storm at the Facebook event.

The Storm-on-Hadoop work is among the first of many promised improvements to come thanks to YARN, a major update to the Apache Hadoop 2.0 code that lets Hadoop clusters run multiple processing frameworks simultaneously. Twitter has been using the open source Mesos resource manager to achieve the same general capabilities, but Gade’s colleague Dmitriy Ryaboy said during the same talk that the company is switching to YARN because it expects — probably correctly — much more community effort will go toward continuously improving its capabilities.

Facebook’s Wormhole project isn’t open source (as far as I can tell), but its lessons are still valuable (and LinkedIn has open sourced a similar technology named Kafka). It’s what’s called a publish-subscribe system, which is essentially a concise way of saying that it manages communications between applications that publish information (e.g., updates to a database) and subscribe to the information their fellow applications are publishing. At Facebook, for example, Wormhole sends changes to Facebook’s master user database to Graph Search so that search results are as up to date as possible, or to its Hadoop environment so analytics jobs have the newest data.

Of course, like all things Facebook (its new Presto interactive query engine comes to mind), Wormhole is built to scale. Latency is in the low milliseconds and, blog post author Laurent Demailly notes

“Wormhole processes over 1 trillion messages every day (significantly more than 10 million messages every second). Like any system at Facebook’s scale, Wormhole is engineered to deal with failure of individual components, integrate with monitoring systems, perform automatic remediation, enable capacity planning, automate provisioning and adapt to sudden changes in usage pattern.”

Although they were developed within separate companies, there’s actually a tie that binds Yahoo’s Storm-in-Hadoop work and Facebook’s Wormhole. As web companies grow from their initial applications into sprawling business composed of numerous applications and services, so too do their infrastructures. To address the differing needs of their various systems at the data level, the companies have begun breaking them down by their latency requirements (i.e., real-time, near real-time and batch, however they choose to word them) and then building tools such as Storm and Wormhole to manage to flow of data between the systems.

We’ve previously explained in some detail how LinkedIn and Netflix have built their data architectures around these principles, and we’ll hear a lot more about how they and other web companies are tackling this situation at Structure next week. Among the speakers are senior engineers and technology executives from Facebook, Google, LinkedIn, Box, Netflix and Amazon.

Feature image courtesy of Shutterstock user agsandrew.

It happened a little quicker than we thought, but AppFog and its Platform-as-a-Service will be part of CenturyLink’s Savvis cloud lineup.

Terms of the acquisition, first reported by GigaOM, were not disclosed, but in a statement, CenturyLink said the AppFog public PaaS will be offered via its savvisdirect online catalog that offers an array of hosted services. Savvis will also offer private, dedicated AppFog deployments to big customers.

AppFog CEO Lucas Carlson (pictured above) is now vice president, cloud evangelist for Savvis.

This acquisition is part of a bigger trend in which PaaS providers are trying to sell big companies on this model to develop and deploy their custom corporate applications but traction has been limited.

Software-as-a-Service took off as Salesforce.com provided a strong argument for moving all the capex spending from on-premises CRM and salesforce automation applications to a subscription model that is paid out of operational expenses. Infrastructure-as-a-service took developers by storm when they saw they could bypass IT to spin up and down development sandboxes cheaply as needed. But many big companies still want their corporate apps to run in- house, even if they’re developed outside.

But lots of vendors continue to push the corporate PaaS case. Red Hat just made its OpenShift PaaS generally available and Pivotal launched a re-architected Cloud Foundry V.2 as a paid-only service. Some expect that other hosting providers, all of which are trying to build credibility with cloud services, will follow CenturyLink/Savvis footsteps and buy up remaining independent public PaaS providers.

Tickets On Sale Now

Simplicity is the ultimate sophistication.

And if simplicity is the ultimate sophistication, iOS 7 is very, very sophisticated.

The most powerful man in the room doesn’t need a loud voice. The securely rich don’t need to flaunt their wealth. The cool kids are the ones who aren’t trying to be cool. The truly elegant realize that understatement is the essence of sophistication.

iOS 7 camera app

And the ultimate interface is the least obvious to the user.

The latest version of iOS is the culmination of Apple’s six-year journey with iPhone, and that journey mirrors the changes that successfully brought Apple back from the brink of death in the late 1990s. The product that saved Apple was the iMac, and Steve Jobs threw it like a grenade into the the computer industry. Where the industry was grey, iMac was color. Where the industry was separate pieces wired together, iMac was unified and singular. Where the industry was opaque, iMac was translucent.

But the original iMac was also a little childish — a little young — before it became the singular objét d’art that it is today. And like iMac’s 15-year transition from fruity to elegant, iOS has now crossed the user interface Rubicon from Microsoft Bob to the Bauhaus — from decoration to design.

Versions 1 through 6 were Apple’s mobile juvenilia; iOS 7 is Apple at the zenith of its design zen.

Less inter, more face

By definition, an interface is a bridge. A connection. Something that takes one thing and attaches it to another. A user interface takes what we want out of the machine and supplies it to us in a format we can see, understand, and use.

Siri in iOS 7

Glancing back at former iOS versions after using iOS 7 is an interesting experiment. As revolutionary, incredible, and praised as it was, old iOS seems obvious, heavy-handed, and almost crude in comparison. Buttons are massive, messy objects that just beg to be interpreted as old-world meatspace buttons that physically depress with a ker-thunk! as you hit them. Borders surround chunks of information like arm-in-arm security staff holding back the outside world from some very exclusive event.

Now, iOS 7 is much more authentically digital and much less representationally physical.

We’ve all grown up, Apple is saying, and we no longer need these real-world touchable referents in order to understand digital information. And, in fact, they’re holding us back, restraining us and surrounding the content we want with gimmick and gewgaw.

Who among us has ever used a rolodex? Do you keep a pad of paper handy at all times to jot down notes with pen and ink? Do we really need green felt, as Apple’s Federighi joked about in the recent WWDC keynote, to make us think of games and fun? And do we all have smoking jackets, cee-gars, and snifters brimming with brandy as we amble back to the snooker table, one hand in our houndstooth jackets?

No, no, and no.

One of the key values in iOS 7, Apple says, is deference. The user interface is there to help users get, understand, and work with the data and information they want, not to compete with it. Interface should be underneath, Apple is saying. It’s kingmaker, not king; servant, not master. Background, not foreground.

Elegant doesn’t mean somber

In spite of its elegance, iOS 7 isn’t standoffish or unapproachable. Not dark and distant.

Rather — and this is why some have likened it to Windows Phone and the new Windows design language — iOS 7 is light, approachable, even colorful in places. In other words, Apple’s not trying for an affect here. It’s not trying so hard to make iOS the nth degree in haute couture so that it becomes elitist or arrogant. Rather, Apple is looking for the best way to present and offer up experiences enabled by the device in your hand — a style so unstyled it’s not really a style but an expression of humility to the text, images, data, voices, and fun that will flow through to the user.

And that’s done in living color.

Minimalism is sometimes very minimal

While I’m happy with Apple’s overall direction, let’s keep in mind that this is a beta operating system and not a universally released, fully available product. And there’s room for improvement.

Which button is pressed? Is there a button? Can you press it?

Minimalism is a language that not everyone speaks, and sometimes less is just … less.

Just one example is in Apple’s phone app, which indicates that yes, you are on speaker mode with a very subtle light effect under the volume icon. It’s not nearly as obvious as iOS 6′s speaker mode — which is the point, after all — but the question is, is it obvious enough?

In other words, can you be so simple, so restrained, and so tentative in what you put into your user interface that you’re actually increasing the cognitive load on your users?

In a word, yes, and I think that there are some in the iPhone user camp who are going to be astonished but not delighted with some elements of the new operating system. Microsoft is going through its own challenges as users are being forced through a massive update in its desktop design language, and I think Apple will see something similar.

 Going somewhere good

That said, I like iOS 7. In fact, I really, really like iOS 7.

Photos in iOS 7

I have been using it for the better part of a week now, and despite the bugginess inherent in a beta operating system (sometimes the phone app hangs, sharing photos and videos doesn’t work in all cases, and so on), I would not go back.

We give bright, noisy toys to kids to hold their attention, and juveniles drive loud, flashy cars to reinforce the image of themselves that they desire to project, but we don’t need that.

We’re all grown up — mostly — and we know that simplicity is the core of sophistication.

Apple knew that back in the 1970s, when Steve Jobs adopted that as one of the company’s marketing slogans. Leonardo da Vinci knew it when he first said it.

And, I think, most Apple fans know it, too.

Image credits: Most: Brian Roizen, top, John Koetsier

Tickets On Sale Now

President Obama is trying to get off the tech community’s shit list.

The President issued a memorandum today to expand the availability of spectrum and bolster America’s leadership in wireless innovation. He mandated that federal agencies free up a significant portion of wireless spectrum so that it can be used by individuals and businesses, and he also announced that the government will invest $100 million in research and development of spectrum-sharing technology.

The initiatives are in an effort to fuel American innovation and maintain the country’s position as a “global leader in wireless broadband technologies.”

“These new initiatives are the latest in a series of actions the administration has taken over the past four years to ensure American businesses and workers have the infrastructure they need to compete in the 21st-century economy,” said a fact sheet the administration issued this morning. 

The President issued a memorandum in 2010 called “Unleashing the Wireless Broadband Revolution,” which required that 500 MHz of spectrum be made available for commercial use within the next 10 years. However, the percentage of American homes reached by high-speed broadband networks has more than quadrupled since 2009 and is now at 80 percent. Use of mobile devices has exploded in the past few years, putting greater strains on available spectrum and making the need to increase access more urgent.

The military, government agencies, and law enforcement currently use a significant portion of available spectrum (which is a finite resource) for national security and public safety. In July 2012, the President’s Council of Advisors on Science and Technology presented a report that said “it is imperative we make enough wireless spectrum available to meet the needs of rapidly expanding and innovative sector of the economy.” Increased access will not only spur domestic economic growth but also help keep the U.S. on top of the technological hierarchy.

President Obama has been a vocal advocate for technical innovation and emphasized over and over the important role that entrepreneurship plays in driving the U.S. economy. Last week, he announced an ambitious plan to get 99 percent of American students connected to lightning-fast Internet within five years. He said that American schools, where only 20 percent of students have access to high-speed Wi-Fi, are falling behind nations like South Korea, where 100 percent of students are wired. Furthermore, the President has made a major push for STEM (science, technology, engineering, math) education. American students’ test scores in this areas lag far behind those of other countries, and there are far more open STEM jobs than qualified people to fill them.

The spectrum initiative is part of the overall mission to grow the U.S. economy through technology. The announcement comes a week after news broke about the FBI and NSA’s top secret data-mining project, PRISM. The snowball of privacy invasion keeps on growing, and today Bloomberg came out with a report that found that thousands of technology, finance, and manufacturing firms are sharing customer data with the government.

The government may be spying on your Internet activity, but it also wants you to have better access to that Internet.

While the expansion of access to spectrum and the improvement of Internet in schools are crucial steps toward supporting technical innovation and the economy, so is establishing an environment where businesses, entrepreneurs, and individuals are not indiscriminately being spied on.

Photo Credit: White House/Flickr

This submersible is barely the size of two soda cans, yet scientists think it might be useful for finding life on and around other planets, including Jupiter’s Europa, an ice-covered moon.

The nano-sub is called DADU and is a project of Uppsala University’s Division of Microsystems Technology. Created by researchers Jonas Jonsson, Erik Edqvist, Hugo Nguyen and Greger Thornell, the submarine is designed to explore subglacier lakes, the likes of which exist at Earth’s own poles and also host microbial life capable of surviving at extremely low temperatures.

The European Space Agency is planning a mission to measure the thickness of surface ice on Europa in 2030, and NASA is also planning a Europa orbit mission. While specific instruments haven’t yet been designated, the DADU prototype shows promise for such missions.

The project was first created as part of a partnership between NASA and the Swedish university. Nano satellites have proven useful for gathering data in low-cost missions.

Europa’s surface is covered in ice, so the scientists think a robotic mission might be able to uncover more data by drilling into the moon’s surface and exploring what lies beneath.

From a brief on the submarine and its intended original use case on Earth:

Some of these lakes have been sealed off from their surrounding environments for considerable periods of time and are thought to harbor a unique biota, and contain important information on how life conditions were several thousand years ago. The DADU project [objective] was to develop a miniature submersible vehicle, which could perform distant and autonomous exploration, and analysis in harsh and narrow environments. It was designed to be able to reach down through narrow glacial bore holes and into the previously unreachable subglacial lakes, to explore and make in-situ measurements.

The sub carries thrusters for navigation, a miniaturized side-scan sonar for subglacial imaging, a fiber-optic cable for real-time communication, a camera, an aquatic sampler, and other instruments for data collection.

Image credit: ASTC

REUTERS/Chip East

Google founders Larry Page and Sergey Brin are recognized for their efforts at the conclusion of the Clinton Global Initiative in New York, September 22, 2006. Former US President Bill Clinton’s annual event brings together world leaders from business, government and philanthropy to try to solve world issues.

See Also

Thousands of companies work closely with U.S. national security agencies by swapping sensitive trade information for benefits including access to classified intelligence, four people familiar with the process told Michale Riley of Bloomberg.

The sources said the information is used not only to defend the nation but also to help the NSA hack the computers of U.S. adversaries.

The reports sheds light on the remarkably close relationship between private technology and finance companies and the government.

In exchange for their cooperation, leaders of companies are given classified information or warnings about threats that could affect their bottom line (such as serious cyebrattacks and who is behind them).

In one instance, according to one of the sources, Google co-founder Sergey Brin was given a temporary classified clearance to see highly sensitive government intelligence linking a cyberattack on the tech giant to a specific unit of China’s military.

The report raises questions about how much and what kind of information is given to the government by companies, especially in light of NSA whistleblower Edward Snowden’s revelation that the government gives America’s tech giants orders that “serve as one-time blanket approvals for data acquisition and surveillance on selected foreign targets for periods of as long as a year.”

One source said that U.S. telecoms voluntarily provide intelligence agencies with access to facilities and data offshore that would require a judge’s order if it were done in the U.S.

Riley writes that the extensive cooperation “is legal and reaches deeply into many aspects of everyday life,” but receives little scrutiny. The companies are reportedly guaranteed immunity from civil actions resulting from the exchanges.

Experts say that lawmakers seeking oversight over the highly-technical programs are in over their heads.

“That’s what makes this issue of oversight so challenging,” Jacob Olcott, a former cybersecurity assistant for Senator John D. Rockefeller IV (D-WV) and now a principal at Good Harbor Security Risk Management. “You have a situation where the technology and technical policy is far outpacing the background and expertise of most elected members of Congress or their staffs.”

SEE ALSO: 
After Classified Briefing Lawmaker Says NSA Revelations Are Only ‘The Tip Of The Iceberg’

World Economic Forum via flickr

Bill Gates and Warren Buffett have been buddies for more than two decades.

When you’ve been friends that long, you’re going to relax and have fun whenever you get together, right? 

At Berkshire Hathaway’s annual shareholders meeting last month, Gates and Buffett played ping-pong and had a “newspaper-throwing contest,” Gates, who joined LinkedIn Thursday, said in his first post. 

Turns out they also had a newspaper-throwing contest last year. Here’s the video: 

In his LinkedIn post, Gates also talked about the influence Buffet has had on him since he first met the legendary billionaire investor in 1991. 

Gates confessed to being not-so-thrilled by the prospect of meeting him. 

Gates figured Buffett cared about nothing else but investing, and was surprised when Buffett started peppering him with business related questions about IBM and Microsoft.

“That’s when I realized he thought about business in a much more profound way than I’d given him credit for,” Gates said in the post. 

Buffett’s annual letter to Berkshire Hathaway shareholders, which is famous for its mix of financial data and philosophical wisdom, inspired Gates to write his own annual letter.

But Gates says his letter is still a work in progress.

“I still have a ways to go before mine is as good as Warren’s, but it’s been helpful to sit down once a year and explain the results we’re seeing, both good and bad,” Gates said in the post.

SEE ALSO: 
Microsoft Is So Sure You’ll Like Windows 8 If You Try It That It’s Invading A Best Buy Store Near You

Apple

See Also

Soon you’ll be able to zoom in while shooting a video on your iPhone.

The new feature will be part of iOS 7, the new version of Apple’s operating system for iPhones and iPads that will be available later this year. We first read about the video zooming feature on iDB.

Zooming in during video capture can be done with the classic pinching gesture. A simple tap will auto-focus.

And remember, it’s already possible to take a photo while filming a video on your iPhone. Just tap the white button located on the left of your screen.

In case you hadn’t heard yet, we’re hosting a cool mini-conference in the beautiful capital of Catalonia on the evening of Thursday 11 July.

We’re organizing the event in collaboration with the local tech community brewers from Barcelona.IO and the fine beer brewers at Estrella Damm.

TNW @ Barcelona – the details

Date and time: Thursday 11 July, from 7PM to 9PM (followed by a meetup with drinks for everyone)

Venue: Antiga Fàbrica Estrella Damm (C/ Rosselló 515, Eixample district – a short walk from the awesomeness that is Sagrada Família)

Where can I register? Right here.

We’re still working on the final format and speaker line-up, but these are some of the people you will be hearing from:

Martin Mignot: Principal and early-stage investor at Index Ventures (involved in companies like SoundCloud, Seedcamp, HouseTrip and Erply)

Andrés Bou: Co-founder of social gaming phenomenon Social Point

Marcel Rafart: Founding partner of VC firm Nauta Capital, executive board member of both Privalia and Groupalia

Robin Wauters (aka, me): European Editor of the Next Web

Vicenç Martí: CEO of that other social gaming phenomenon, Akamon

Ángel García: Founding partner of Lanta Digital Ventures

Jochen Doppelhammer: Founder and CEO, Yuilop

We will also be doing something fun to highlight some very early-stage yet promising Barcelona startups – more on that later.

We’ll be adding more speakers over time and will post another update once we have more details to share on both format and line-up.

Until Friday next week, you can register for 15 euros; after that the price will be 20 euros. Note that this includes entrance for the event and as much Estrella Damm beer as you want afterwards as you discuss which keynote or panel you liked best. If you insist, we’ll also serve water.

The original cut-off date for the early-bird tickets was today, but since we’re a tad late announcing our first speakers for the event (mea culpa) we’ve decided to extend it until next week. Note that the capacity of the venue we booked is capped at 250 seats, so space will be limited.

We hope you register and are looking forward to seeing you!

Image credit: Thinkstock

The Oxford English Dictionary (OED) has been updated to include 1,200 new words and word revisions used commonly in tech circles, such as tweet and big data .

While terms such as tweet, follow and followers were already in the OED, their usage has been expanded to include use in a social media context.

In fact, in a break with tradition, the inclusion of tweet as a noun and a verb breaks one of the OED’s own rules – namely that a word has to be in use for at least 10 years before it’s considered for inclusion, but as John Simpson, chief editor of the OED, notes “it seems to be catching on”.

The June quarterly update to the OED  includes a number of technology additions such as big data, crowdsourcing, e-reader, mouseover, redirect (the noun), and stream as a verb.

While only a small proportion of the 1,200 changes or additions, are specifically tech words, terms such as geekery in the sense of behavior relating to a geek (rather than referring to “bizarre or grotesque acts performed at carnivals or circuses” as it did in its first incarnation).

Also making the list for the update are terms such as flash mob  as which refers to a group of people “who assemble in public to perform a pre-arranged action together and then quickly disperse”, which you – but not the OED – probably already knew.

Other notable additions for the quarterly update include binge drinking and binge drinker, BFF (Best Friend/Friends Forever), dad dancing, fiscal cliff, to have a cow, and pay day loan. It also includes cludgie, which is slang for toilet, apparently.

Image Credit – Thinkstock/Getty Images