Dave Holtz

Dave Holtz

PhD Student in Information Technology at MIT Sloan School of Management

Sitemap

A list of all the posts and pages found on the site. For you robots out there is an XML version available for digesting as well.

Pages

Posts

Blog Post number 4

less than 1 minute read

Published:

This is a sample blog post. Lorem ipsum I can’t remember the rest of lorem ipsum and don’t have an internet connection right now. Testing testing testing this blog post. Blog posts are cool.

Blog Post number 3

less than 1 minute read

Published:

This is a sample blog post. Lorem ipsum I can’t remember the rest of lorem ipsum and don’t have an internet connection right now. Testing testing testing this blog post. Blog posts are cool.

Blog Post number 2

less than 1 minute read

Published:

This is a sample blog post. Lorem ipsum I can’t remember the rest of lorem ipsum and don’t have an internet connection right now. Testing testing testing this blog post. Blog posts are cool.

Blog Post number 1

less than 1 minute read

Published:

This is a sample blog post. Lorem ipsum I can’t remember the rest of lorem ipsum and don’t have an internet connection right now. Testing testing testing this blog post. Blog posts are cool.

portfolio

publications

A study of the trace 39Ar content in argon from deep underground sources.

Published in Astroparticle Physics, 2015

The discovery of argon from deep underground sources with significantly less 39Ar than atmospheric argon was an important step in the development of direct dark matter detection experiments using argon as the active target. We report on the design and operation of a low-background single-phase liquid argon detector that was built to study the 39Ar content of this underground argon. Underground argon from the Kinder Morgan CO2 plant in Cortez, Colorado was determined to have less than 0.65% of the 39Ar activity in atmospheric argon, or 6.6 mBq/kg specific 39Ar activity.

Recommended citation: Xu, J., et al. "A study of the trace 39Ar content in argon from deep underground sources." Astroparticle Physics 66 (2015): 53-60.

Limiting Bias From Test-Control Interference in Online Marketplace Experiments

Published in Workshop on Information Systems and Economics (WISE 2016), Dublin, Ireland, 2016

Many internet firms use A/B tests to make product decisions. When running an A/B test, the typical objective is to measure the average treatment effect (ATE), i.e. the difference between the average outcome in the counterfactual situation where 100% of users are exposed to the treatment and the average outcome in the status quo situation (in which 100% of users are exposed to the control). However, a simple difference-in-means estimator will give a biased estimate of the ATE when outcomes of units in the control depend on the behavior or outcomes of units in the treatment - a case referred to in this work as test-control interference. Both Blake and Coey [2] and Fradkin [6] find evidence of bias due to test-control interference in online marketplace experiments. This paper considers the use of experimental designs and ATE estimators discussed by Eckles et al [5], which reduce bias in the presence of peer effects, in online marketplace experimentation. In order to do so, I model the marketplace as a network in which an edge exists between two sellers if their goods substitute for one another. Using data scraped from Airbnb [16], I create an agent-based simulation to model seller outcomes, both under the status quo and subject to “treatments” that force hosts to lower their price. Having estimated hosts’ baseline outcomes and their outcomes when 100% of hosts are subject to the treatment, I use the same simulation framework to approximate ATE distributions obtained when various network experiment design and analysis techniques are used. I find that graph cluster randomization leads to bias reductions of as much as 62%. Unfortunately, the variance of ATE estimators also increases significantly. Replacing the simple difference-in-means estimator with more sophisticated ATE estimators can lead to mixed results. While some methods (i.e., exposure models) provide (small) additional reductions in bias and small reductions in variance, others (i.e., the Hajek estimator for the ATE) lead to increased bias and variance. Although further work is needed, current results suggest that experiment de- sign and analysis techniques from the network experimentation literature are promising tools for for reducing bias due to test-control interference in marketplace experiments.

Recommended citation: Holtz, David. “Limiting Bias From Test-Control Interference in Online Marketplace Experiments.” Workshop on Information Systems and Economics (WISE 2016), Dublin, Ireland. 2016.

The determinants of online review informativeness: Evidence from field experiments on Airbnb.

Published in Working Paper, 2017

Reputation systems are used by most digital marketplaces but their design varies greatly across websites. We use the setting of Airbnb to study how design choices affect the ability of ratings and reviews to aggregate information. We study two experimental changes to the reputation system of Airbnb. The first change offered guests a $25 coupon to submit a review. The second change implemented a simultaneous-review system, which eliminated strategic reciprocity from reviews. We show that both experiments made the reputation system more informative and use our findings to quantify the importance of mechanisms that cause inefficiency in reputation systems.

Recommended citation: Fradkin, Andrey, Elena Grewal, and David Holtz. The determinants of online review informativeness: Evidence from field experiments on Airbnb. Working Paper, 2017.

talks

teaching

Teaching experience 1

Undergraduate course, University 1, Department, 2014

This is a description of a teaching experience. You can use markdown like any other post.

Teaching experience 2

Workshop, University 1, Department, 2015

This is a description of a teaching experience. You can use markdown like any other post.