From Google to browser bookmark menus, collections of information resources are
typically presented in a form which is impoverished for cognition and
experience: the list of textual elements.
We need to represent collections of information resources in
personal forms that help us discover connections between the resources.
combinFormation is a mixed-initiative tool for browsing and collecting,
which uses sampling and composition to make connections
between information resources visible.
Information composition is a holistic, integrative representation for information collections, in which a set of visual semantic clippings and annotations is arranged to form a connected whole.
Some examples by students in ENDS 101, are here. The CreativeIT canon compositions are here.
InfoComposer is a free and open source interactive system that uses composition for collecting, arranging, annotating and browsing image and text clippings from web pages. The samples act as visual and semiotic surrogates for the documents from which they are extracted, supporting navigation back to the source. Rich metadata, like the authors, categories, and citations of articles and patents, and the price, ratings and store sections of Amazon products, is extracted for each clipping source. Compositing operations on image clippings can be used to create a visual sense of connectedness among adjacent elements.
A prior information composition system, combinFormation uses mixed-initiative information composition. Mixed-initiative systems combine pro-active software agents, which work in cooperation with the user, with direct work by the user. combinFormation's initiatives are the system's generation of composition, and the user's direct manipulation. The system's generative actions -- collecting information samples, and composing them visually -- are conducted iteratively, based on a user model. The system presents the ongoing generation of the composition to the user in an interactive information space. In this space, one of the user's initiatives is to directly manipulate the composition through interactive design operations, which enable samples to be displaced, layered, annotated, and removed. The user can also express positive or negative interest in each sample. Expressions of interest affect the model, creating a feedback loop through the visualization.
publications
Webb, A. and Kerne, A.,
Creative Visual Thinking through Information Composition + Diagramming,
Visual Thinking & Digital Imagery Workshop, CHI 2012, Austin, TX, USA.
Linder, R. and Kerne, A.,
"Using Information Composition to Represent Connections Among Events Across Time and Place",
Heritage Matters: Designing for current and future values through digital and social technologies, CHI 2012, Austin, TX, USA.
Webb, A.M., Kerne, A.
Integrating Implicit Structure Visualization with Authoring Promotes Ideation,
Proc Joint ACM/IEEE Conf. on Digital Libraries (JCDL),
June 13-17, 2011, Ottawa, Canada.
Koh, E., Kerne, A.,
Deriving Image-Text Document Surrogates to Optimize Cognition,
Proc ACM DocEng 2009,
Munich, Germany, Sept 16-18.
Kerne, A., Koh, E., Smith, S. M., Webb, A., Dworaczyk, B.,
combinFormation: Mixed-Initiative Composition of Image and Text Surrogates Promotes Information Discovery
,
ACM Transactions on Information Systems (TOIS), Dec. 2008, 5:1-5:45,http://doi.acm.org/10.1145/1416950.1416955.
Kerne, A., Damaraju, S., Kumar, B., and Webb, A.
Meta-Metadata: A Semantic Architecture for Multimedia Metadata Definition, Extraction, and Presentation.
,
Poster and Demo Proceedings of the 3rd International Conference on Semantic and Digital Media Technologies,
Koblenz, Germany, 3-5 December 2008.
Kerne, A., Koh, E.,
Representing Collections as Compositions to Support Distributed Creative Cognition and Situated Creative Learning,
New Review of Hypermedia and Multimedia (NRHM),
13(2) Dec 2007, 135-162.
Kerne, A., Smith, S.M., Koh, E., Choi, H., Graeber, R.,
An Experimental Method for Measuring the Emergence of New Ideas in Information Discovery,
International Journal of Human-Computer Interaction (IJHCI),
24 (5) July 2008, 460-477.
Webb, A. and Kerne, A.
The In-Context Slider: A Fluid Interface Component for Visualization and Adjustment of Values while Authoring,
Proceedings of ACM Advanced Visual Interfaces,
Napoli, Italy, 28-30 May 2008.
Koh, E., Kerne, A., Webb, A., Damaraju, S., Sturdivant, D.,
Generating Views of the Buzz: Browsing Popular Media and Authoring using Mixed-Initiative Composition,
Proceedings of ACM Multimedia,
Augsburg, Germany, 23-29 September 2007.
Webb, A.,
A Transitory Interface Component for the In-Context Visualization and Adjustment of a Value
,
M.S. thesis, Texas A&M University, August 2007.
Kerne, A., Koh, E., Dworaczyk, B., Mistrot, J.M., Choi, H.,
Smith, S., Graeber, R., Caruso, D., Webb, A., Hill, R., Albea, J., A
Mixed-Initiative System for Representing Collections as Compositions of
Image and Text Surrogates,Proc Joint ACM/IEEE Conf. on Digital Libraries (JCDL), June 2006, Chapel Hill.
Kerne, A., Koh, E., Smith, S.M., Choi, H., Graeber, R., Webb., A.,
Promoting Emergence in Information Discovery by Representing Collections with Composition,
Proc ACM Creativity & Cognition,
Washington DC, June 2007, 117-126.
Koh, E., Kerne, A.,
combinFormation: Exploring Multiple Searches Together through the Mixed-Initiative Composition Space,
Proc ACM Computer Human Interaction 2007 Workshop on Exploratory Search and HCI,
San Jose, April 2007.
Koh, E., Caruso, D., Kerne, A., Gutierrez-Osuna, R.,
Elimination of Junk Document Surrogate Candidates through Pattern Recognition,
Proceedings of ACM Symposium on Document Engineering,
Winnipeg, Canada, 28-31 August 2007.
Koh, E., Kerne, A., Hill, R.,
Creativity Support: Information Discovery and Exploratory Search,
Proceedings of ACM SIGIR Conference,
Amsterdam, Netherlands, 23-27 July 2007.
Kerne, A., Koh, E., Choi, H., Dworaczyk, B., Smith, S.M., Hill,
R., Albea, J., Supporting
Creative Learning Experience with Compositions of Image and Text
Surrogates, Proc Ed Media 2006, June 2006, Orlando.
Kerne, A., Koh, E., Sundaram, V., Mistrot, J.M.,
Generative
semantic clustering in spatial hypertext, Proc ACM Document
Engineering 2005,
Nov 2005, Bristol, UK, 84-93.
Kerne, A., Mistrot, J.M., Khandelwal,
M.,Sundaram, V., Koh, E., Using
Composition to Re-Present Personal Collections of Hypersigns, Proc
Computational Semiotics in Games and New Media (CoSIGN), Sept 2004,
Split, Croatia.
Kerne, A., Smith, S.M., Mistrot, J.M., Sundaram, V., Khandelwal, M., Wang, J., Mapping Interest and Design to Facilitate Creative Process During Mixed-Initiative Information Composition, Proc Interaction: Systems, Practice and Theory, a Creativity & Cognition Symposium, 2004, 1-25.
Kerne, A., Sundaram, V., A Recombinant Information Space, Proc Computational Semiotics in Games and New Media (CoSIGN) 2003, 48-57.
Karadkar, U.P., Kerne, A., Furuta, R., Francisco-Revilla, L., Shipman, F., Wang, J., Connecting Interface Metaphors to Support Creation of Hypermedia Collections, Proc European Conf Digital Libraries 2003, 338-349.
Kerne, A., Concept-Context-Design: A Creative Model for the Development of Interactivity. Proc ACM Creativity and Cognition 2002, 192-199.
Kerne, A., CollageMachine: Interest-Driven Browsing Through Streaming Collage. Proc Cast01: Living in Mixed Realities. Bonn, Germany, 241-244.
Kerne, A., CollageMachine: An Interactive Agent of Web Recombination, Leonardo 33:5, 2000.
Kerne, A., CollageMachine: Temporality and Indeterminacy in Media Browsing via Interface Ecology. Proc CHI 1997: Extended, 297-298.
Kerne, A., Smith, S.M., Choi, H., Graeber, R., Caruso, D.,
Evaluating Navigational Surrogate Formats with Divergent Browsing Tasks, Proc ACM CHI 2005.