Type
Text
Type
Dissertation
Advisor
Ramakrishnan, I.V. | Borodin, Yevgen | Ramakrishnan, C.R. | Warren, David | Bigham, Jeffrey.
Date
2013-12-01
Keywords
accessibility, automation, blind, macro, non-visual, web | Computer science
Department
Department of Computer Science.
Language
en_US
Source
This work is sponsored by the Stony Brook University Graduate School in compliance with the requirements for completion of degree.
Identifier
http://hdl.handle.net/11401/77309
Publisher
The Graduate School, Stony Brook University: Stony Brook, NY.
Format
application/pdf
Abstract
The Web is far less usable and accessible for people with visual impairments than it is for sighted people. There are only a handful of screen-reader applications that improve web browsing accessibility by reading aloud the content of web pages. However, accessibility does not imply usability; blind people end up spending minutes on browsing tasks that take sighted people seconds. This lack of productivity using the Web poses a significant barrier for people with visual impairments in education, employment, healthcare, etc. Web automation, a process of automating browsing actions on behalf of the user, has the potential to bridge the divide between the ways visually impaired people and sighted people access the Web. Typical automation interfaces require that the user record or handcraft a macro, a set of instructions for automating a useful sequence of browsing actions, so that these actions can be replayed in the future. In this dissertation, I present a novel approach to Web automation that completely automates those tasks, while simultaneously making the web automation easier to control. The first pillar of the presented approach is a novel model-based algorithm that uses the past browsing history and the current web page as the browsing context to predict the most probable browsing actions that the user can do. The model is continuously and incrementally updated as the history evolves, thereby, ensuring the predictions are not " outdated" . A unique aspect of the model is that its construction is " unsupervised" , i.e. does not require any kind of labeled training data. The second pillar is a novel interface that lets the user focus on the objects associated with the most probable predicted browsing steps (e.g. | clicking links and filling out forms), and facilitates automatic execution of the selected steps. Several studies blind participants showed that the proposed approach dramatically reduced the interaction time needed to accomplish typical browsing tasks, and the user interface was perceived to be much more usable than the standard screen-reading interfaces. | 91 pages
Recommended Citation
Puzis, Yury, "Accessible Web Automation" (2013). Stony Brook Theses and Dissertations Collection, 2006-2020 (closed to submissions). 3130.
https://commons.library.stonybrook.edu/stony-brook-theses-and-dissertations-collection/3130