Research Overview: Aaron Hertzmann

Broadly speaking, I am interested in all areas of computer graphics and computer vision. My work seeks to address the following high-level questions:

Moreover, I am interested in applications of Machine Learning to these two areas, as well as applications to Human-Computer Interaction.

Some specific research areas:

Image and video stylization
How can we write computer software that helps in creating artistic imagery and video? How can we enable computer animation in the styles of human painting and drawing?

Painterly rendering Painterly video Image Analogies Paint By Relaxation
Curve Analogies Segmentation-Based NPR Interactive painterly animation PortraitSketch
Controlling Neural Style Im2Pencil Multidomain Stylization Video Rigidification

Stylization of 3D models
Line drawing and occluding contour algorithms for 3D models, for stylization and art.

Illustrating smooth surfaces Artistic Stroke Thickness Learning Hatching Computing smooth contours
Line Drawings from 3D Neural Contours Neural strokes Accurate Occluding Contours
Algebraic Contours Contour Insights Region-based simplification

Art, AI, Perception Essays
See also my blog

The Science of Art Can Computers Create Art? Visual Indeterminacy Why Do Line Drawings Work?
Computers Do Not Make Art Edges in Drawing Perception Toward modeling creativity Choices in Photography
Generative AI Theory of perspective

Graphic Design and Data-Driven Aesthetics

Learning Label Layout Color Compatibility Learning Single-Page Layout Color personalization
Clip art style similarity Font attributes Recognizing Image Style Infographics style
Interactive layout suggestions Illustration datasets Visual design importance Behance Artistic Media
Stroke-Based Fonts Context-Aware Asset Search LayoutGAN Visual Font Pairing
Design and photo importance Quantifying art ambiguity Indeterminate aesthetics

Learning Human Motion Models from Data
How can we create virtual characters from live human performance data?

Style machines Style IK Learning biomechanics Motion Composition
Shared Latent Gaussian Processes Gaussian Process Dynamical Models Style-Content Gaussian Processes Active learning for mocap

Controllers for simulated locomotion
Techniques for creating human and animal motor controllers that move in physically-realistic and expressive ways, inspired by insights from biology, robotics, and reinforcement learning.

Prioritized Optimization Optimizing Walking Walking with Uncertainty Feature-Based Controllers
Low-Dimensional Planning Full-Body Spacetime Rotational control

Person tracking and reconstruction from video
How do we perceive the 3D structure of a video sequence that contains moving people and objects?

Non-rigid SFM Automatic non-rigid shape from video Rotoscoping Kinematic person tracking
Physics-Based Person Tracking Hand reconstruction Contact and dynamics estimation Contact-aware retargeting
HuMoR motion model

Machine learning for geometry processing

Surface Texture Synthesis Learning body shape variation Real-Time Curvature Learning mesh segmentation
Furniture style Metric Regression Forests Learning segmentation from scraping

Virtual reality video and interfaces

VR Video Editing VR Video Review Depth Conflict Resolution VR Widgets
6-DoF VR video View-Dependent VR Video

Rigid shape reconstruction

Smooth surfaces from video Example-based photometric stereo Example-based multiview stereo Scanning with varying BRDFs
Image-Based Remodeling

Image Understanding, Photo Editing, GANs

Single-image deblurring Image Sequence Geolocation Acceptable photographic adjustments Deep image tagging
Portrait Segmentation GAN projection GANSpace ZoomShop

Other machine learning and data science

Segmental speech processing Latent Factor Travel Model Sparse Gaussian Processes Event sequence visualization
Behance recommendations Toward Better User Studies Curse of User Studies
Aaron Hertzmann