{"id":3080,"date":"2024-07-29T17:01:48","date_gmt":"2024-07-29T17:01:48","guid":{"rendered":"https:\/\/publicconsultation.org\/?page_id=3080"},"modified":"2025-04-17T19:46:47","modified_gmt":"2025-04-17T19:46:47","slug":"methodology","status":"publish","type":"page","link":"https:\/\/publicconsultation.org\/methodology\/","title":{"rendered":"Methodology"},"content":{"rendered":"\n

PPC is committed to ensuring that the samples it surveys are as representative of the target population as possible, by using well-tested methods of stratification and quality control. <\/p>\n\n\n\n

In a study comparing PPC\u2019s opt-in sample with other opt-in samples and probability-based samples analyzed in other academic studies, including by researchers at Stanford University and the Pew Research Center, PPC performed better than all the other opt-in samples, and nearly as well as the probability-based samples. That study can be accessed here<\/a>.<\/p>\n\n\n\n

PPC is also committed to full transparency, in line with AAPOR\u2019s Transparency Initiative<\/a>. For each survey release, a full methodology will be included that details the sample collection process, sample characteristics, weighting method, and quality control measures.<\/p>\n\n\n\n

Survey Development<\/strong>
The surveys are programmed online using the Alchemer or Qualtrics platforms.  The logic and skip patterns are rigorously tested.<\/p>\n\n\n\n

Each survey is programmed in English and Spanish, to ensure that most non-native English speakers, who constitute a vital part of the general adult population, are represented.<\/p>\n\n\n\n

Each survey is soft-launched with several dozen respondents, and the preliminary data is tested to make sure the survey is fielding as intended.<\/p>\n\n\n\n

Data Collection<\/strong>
Survey responses are collected directly on the Alchemer and Qualtrics platforms. Only respondents with a provided link can take the survey.  Surveys are taken on a computer or mobile phone. <\/p>\n\n\n\n

PPS ensures that data is collected in adherence to the European Union\u2019s General Data Protection Regulation policies for data privacy and security, as well as the California Consumer Privacy Act (CCPA).<\/p>\n\n\n\n

Sample Collection<\/strong>
Sample collection is managed by QuantifyAI with oversight from PPC.  The selected sample is invited to participate in a specific survey via email invitation, push notification, or SMS for cell phone users. Respondents are offered cash or cash-equivalent incentives to participate in the survey. Incentives are administered by QuantifyAI or the panel companies.<\/p>\n\n\n\n

Sample collection involves pre-stratification and completion quotas to ensure a representative sample of the adult population by primary demographics: age, income, education, sex, and race\/ethnicity. Benchmarks for each demographic are obtained from the most recent US Census Bureau\u2019s American Community Survey (ACS), and the Current Population Surveys (CPS). <\/p>\n\n\n\n

To meet pre-stratification requirements, sample recruitment accounts for differing response rates among demographic groups, to reduce the errors for the final primary demographic benchmarks. Thus, demographic groups that tend to have lower response rates are over-invited, including people of color, younger adults, and those with less formal education. Respondents are required to answer all primary demographic questions so there is no missing data for these benchmarks.<\/p>\n\n\n\n

Completion quotas are also used to meet pre-stratification requirements, which disqualify respondents based on their primary demographic characteristics once the benchmark for a particular demographic has been met.<\/p>\n\n\n\n

The target population, number of respondents, overall response rate, and confidence interval, as well as the pre-stratification schema, are made available in each survey release.<\/p>\n\n\n\n

Post-Stratification (Weighting)<\/strong>
The post-stratification schema used is made available in each survey release.<\/p>\n\n\n\n

Quality Control<\/strong>
The sample management firm used by PPC \u2013 QuantifyAI \u2013 uses quality control measures to disqualify duplicate responses and survey bots from taking the survey. They check respondents\u2019 IP addresses to determine if there are duplicate respondents, and utilize AI to employ an \u201coperating system & Web browser check\u201d to determine if there are any cross-panel duplicates. To disqualify survey bots, they use the bot-detection software hCaptcha, which estimates human likeness by analyzing IP and cursor behavior.<\/p>\n\n\n\n

Each survey includes standard attention-check questions used industry-wide, as well as honesty questions that are being used more frequently within survey research (e.g. What have you done in the past week? Won a gold medal at the Olympics; Watched TV; etc.). Respondents who fail these tests are disqualified. Each survey also includes a speed limit, which disqualifies respondents early in the survey who are moving at a pace roughly triple the average reading speed. The post-survey sample is also analyzed to remove respondents that answered less than half the substantive questions, or who engaged in straight-lining.<\/p>\n\n\n\n

The quality control measures taken, and the number of respondents disqualified in-survey and removed from the sample post-survey will be made available in each survey release.<\/p>\n\n\n\n

Access to Data<\/strong><\/p>\n\n\n\n

All research by the Program for Public Consultation is archived by\u00a0The Digital Repository at the University of Maryland (DRUM)<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

PPC is committed to ensuring that the samples it surveys are as representative of the target population as possible, by using well-tested methods of stratification and quality control.  In a study comparing PPC\u2019s opt-in sample with other opt-in samples and […]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ngg_post_thumbnail":0,"footnotes":""},"class_list":["post-3080","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/publicconsultation.org\/wp-json\/wp\/v2\/pages\/3080","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/publicconsultation.org\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/publicconsultation.org\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/publicconsultation.org\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/publicconsultation.org\/wp-json\/wp\/v2\/comments?post=3080"}],"version-history":[{"count":3,"href":"https:\/\/publicconsultation.org\/wp-json\/wp\/v2\/pages\/3080\/revisions"}],"predecessor-version":[{"id":4979,"href":"https:\/\/publicconsultation.org\/wp-json\/wp\/v2\/pages\/3080\/revisions\/4979"}],"wp:attachment":[{"href":"https:\/\/publicconsultation.org\/wp-json\/wp\/v2\/media?parent=3080"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}