Are you satisfied?

In introducing its broad e-government agenda last fall, the Bush administration said it wants to use technology to better connect with citizens. Measuring the strength of that connection, however, can be tricky for managers.

NASA and the General Services Administration are conducting pilot studies on Web sites they operate by adding software that offers an online customer satisfaction survey based on the American Customer Satisfaction Index (www.theacsi.org).

For the Web sites, the ACSI index — developed by a partnership of the University of Michigan Business School, the American Society for Quality and the CFI Group, an international consulting firm — will measure the degree to which visitors to the sites perceive them to be useful and effective.

However, some experts believe it takes more than just a survey, with its limited choices, to design a site that meets the public's needs.

The Congress Online Project's recent survey of congressional Web sites, for example, collected data from focus groups and interviews with more than 100 Capitol Hill staffers, as well as through surveys, expert reviews and industry research. A new study of federal Web sites by Andersen urges agencies to evaluate their sites according to best practices recognized in software development circles.

Good design "requires that you spend time talking to users or the representatives of user groups to get a really good sense of the information tasks they have, and then [match] those tasks with the content and information the agency produces," said Steve Fleckenstein, senior manager in Andersen's Office of Government Services, which conducted the study, "A Usability Analysis of Selected Federal Government Web Sites."

Although the methodology behind ACSI has been around since the 1980s, ForeSee Results, which is providing the survey software for NASA's site and GSA's FirstGov portal, only began marketing its product in January, according to Larry Freed, the company's president and chief executive officer.

ForeSee Results' online surveys operate via pop-up windows that open at random for a certain percentage of Web site visitors or via links that any visitor can choose to click. NASA, which uses the pop-up method, asks survey respondents to identify themselves as members of broad social categories, while the FirstGov Customer Satisfaction Survey doesn't ask for such identification.

The NASA Home Page Survey also offers respondents a list of possible additions to the Web site and asks them to choose which one they like best. So far, the NASA Web audience, which consists primarily of students, teachers and the general public, has voted by a 3-1 margin for Webcasts of NASA-TV, the agency's broadcasts of such events as space launches and life aboard the International Space Station, said Brian Dunbar, the Internet services manager in NASA's public affairs office. "What we're doing in this pilot [project] is following up on the president's charge to provide people with the information they want when they want it, instead of just providing the information we think should go on a Web page," Dunbar said.

Focusing on "organizational and political mandates" instead of user needs can lead to sites "that confuse, frustrate and alienate users," the Andersen study concluded.

Other barriers to producing good federal Web sites identified in the study include the agency placing a low priority, or no priority at all, on Web development; assigning Web site tasks to staff members who are not equipped to handle them; maintaining contracting policies that exclude well-qualified Web designers; and having to comply with burdensome federal regulations about gathering information from the public.

In addition, government agencies build far too many portals, the Andersen study said. An interface design model is appropriate for a true portal such as FirstGov, "but it amounts to putting lipstick on a pig for many federal sites [by providing] cover for underlying subsites that still exhibit poor usability," the study said.

***

Designing usable Web sites

How does an agency develop a Web site the public will come back to again and again?

Here are some tips from a new study by Andersen: n Make sure senior executives understand the need for a usable site.

* Centralize and coordinate Web site development and management.

* Balance the needs of the agency and the users.

* Educate agency staff on key user-centered Web design principles.

* Hire contractors who know how to develop usable Web sites.

* Establish a content management system.

* Forget portals.