Documentation

Table of Contents

About the Agricultural Resource Management Survey (ARMS)

The Agricultural Resource Management Survey (ARMS) is sponsored jointly by USDA's Economic Research Service (ERS) and National Agricultural Statistics Service (NASS). ARMS was first conducted in 1996. It combined USDA's previous cropping practices, chemical use, and farm costs and returns surveys, which were conducted separately from 1975 to 1995. ARMS is a multiphase series of interviews with farm operators about their cropping practices, farm businesses, and households.

Phase I–Screening Survey

Farmers and ranchers selected for the survey are contacted to verify that they still qualify as a farm and that they produce the specific commodities targeted by Phase II that year.

Phase II–Production Practices and Costs Report

Phase II questionnaires are only directed to operations producing the survey year’s target crop(s). The target crop is rotated on a multiyear cycle. Data are periodically collected for each major commodity, e.g. corn producers were surveyed in 2001, 2005, and 2010. Common information collected for all surveyed commodities includes (but is not limited to):

  • Field characteristics: field acreage, ownership, seed use and costs, yield, organic practices, conservation practices, crop rotations, and crop insurance.
  • Nutrient or fertilizer applications: applications of synthetic and organic nutrients, including application costs, amounts, methods, and timing; as well as manure, compost, and soil testing practices.
  • Bio-control or pesticide applications, and other pest management practices: applications of biological and chemical pesticides, including costs, amounts, methods, and timing; as well as various pest management practices such as scouting, assessment of pest problems, use of tillage and crop rotation, maintenance of equipment.
  • Field operations: use of machinery and labor in the field. Tillage, technical services, and the use of precision technologies are included in this section. Associated costs for fuel and labor are documented.
  • Irrigation: irrigation practices such as amount of water used, system type, and related expenses.

Phase III–Farm Business and Farm Household Information

Beginning in 2012, the Phase III uses only two questionnaire versions: a general Costs and Returns Report and an expanded Costs and Returns Report for target crop or livestock producers, which collects detailed commodity-specific information. Prior to 2012, a Core (short) version of the questionnaire was used to simplify the general Costs and Returns Report. Information collected on all versions includes (but is not limited to):

  • Land in farm/ranch: acres of land owned or rented, rents paid or received.
  • Acreage and production: acres harvested and total production.
  • Livestock: quantity sold or removed and ending quantity.
  • Commodity marketing and income: quantities and prices associated with marketing contracts, production contracts, cash or open market sales.
  • Other farm-related income: Federal farm programs, custom work, energy lease and royalty payments.
  • Operating and capital expenditures: livestock purchases, feed, seed, fertilizer, chemicals, fuel, labor, taxes, etc.
  • Farm assets: dwellings, farm buildings, land, inventory, equipment.
  • Farm debt: outstanding loans and their terms.
  • Farm management and use of time: farm legal organization, farm business ownership, number of operators, time allocation of operator and spouse.
  • Farm household: age and education of the operator and spouse, nonfarm income, assets, and debt.

Survey Design

ARMS is a multiphase, multiframe, stratified, probability-weighted sampling design. What do these four characteristics of the survey design mean?

Multiphase

ARMS is a series of interviews with farm operators about their management practices, farm business structure and finances, and household characteristics. It is conducted annually in three phases:

  • Phase I: conducted during the summer of the reference year.
    Farmers selected for the survey sample are contacted to verify their operating status and whether they produced the commodities being surveyed that year. Phase I improves survey efficiency.
  • Phase II: conducted in the fall and winter of the reference year.
    Randomly selected operating farms from Phase I are interviewed regarding their production practices and chemical use. Phase II data are collected at the individual field or production unit level and include physical and economic data on production inputs, management practices, and commodity costs of production.
  • Phase III: conducted in the early spring of the year following the reference year.
    A nationally representative sample of farmers, randomly selected from Phase I, is interviewed to collect information on the characteristics and finances of all farm businesses and farm households during the reference year. Phase III data are collected at the whole-farm level and, for the livestock-specific version, at the livestock-enterprise level. As a part of the larger Phase III sample, operators that complete a Phase II questionnaire are also asked to complete a Phase III questionnaire. This provides information on their costs and returns, including data needed to estimate the costs associated with their production practices. The costs of production estimates include the enterprise-share of farm business expense items, such as land taxes, insurance, fuel expenses, etc., collected only in the Phase III interviews.

Diagram of ARMS modular design that connects field practices, financial data, and household demographics

Multiframe

NASS uses two sampling frames to sample a total of about 30,000 farm and ranch operations for ARMS each year. The majority of farms—around 94 percent—come from the List Frame.

  • List Frame—The primary sample is derived from the NASS List Frame. NASS maintains a list of farm operations that exhibit certain characteristics. The list is constructed and maintained from many different sources, including the Census of Agriculture and other NASS surveys. Because some information is already known about these farms, the list can be sorted according to farm types and size classes. For more information on the List Frame, see List Frame Samples on the NASS website.
  • Area Frame—The NASS Area Frame supplements the List Frame. The farm population continually evolves as some farms change ownership or exit and new farms emerge. At any point in time, the List Frame invariably misses some farms, which the Area Frame helps capture. The Area Frame consists of farms captured through a random selection of agricultural land segments. NASS annually conducts a spring survey selected from the Area Frame to estimate crop acreage and land use. For more information on the Area Frame, see Area Frame Samples on the NASS website.

Stratified

Strata are divisions within the sample frames that group farms by region (or by State in the case of 15 heavily sampled States), farm sales category, and commodity specialization. Farms in different strata are sampled with a different probability of selection to ensure that each sample includes a sufficient number of farms in different regions, sizes, and commodities. Within a stratum, the weight (expansion factor) is based on the probability of each sampled unit’s selection.

Probability-weighted

Because the ARMS sample is not a simple random sample, each observation has a different weight, or expansion factor, to reflect its probability of selection and, therefore, what part of the sampled universe it represents. Appropriate sample weights (expansion factors) are provided to prepare population estimates from the survey results. Population estimates are constructed by weighting each sample with the appropriate expansion factor. A jackknife re-sampling process is used with 30 additional weights from NASS for each sample to estimate the variances for each data item.

Furthermore, data from the Phase II of ARMS is divided into three data files: 1) fertilizers, 2) pesticides, and 3) all other data designated as the main file (e.g., field characteristics, management practices, and production input data other than fertilizers and pesticides). Sample weights associated with each of the three data files depend on the number of usable responses for the respective parts of the Phase II questionnaire. The usability of these tables for the construction of chemical or fertilizer use estimates is determined independently from the completion of the remainder of the questionnaire. Typically, slightly different response rates exist for these three parts of the questionnaire, and hence, weights differ between the main file and the two subfiles (pesticide and fertilizer). Cross-tabbing of variables across the three data files can give different population estimates for the same variable. In general, such population estimate differences across tables are relatively small. For more detail, see Farm Production Expenditures Methodology and Quality Measures.

Questionnaires and Manuals

Questionnaires are printed forms or computer programs used to ask specific questions and record the responses given by the sampled respondents. Due to the complex design of the ARMS survey, multiple questionnaires are used each year. Phase I questionnaires are used to collect current information for accurate sampling. All complete Phase II respondents are asked to complete a Phase III follow-on report. Every effort is made to ensure that both the Phase II and Phase III questionnaires are completed for these operations in both samples. Data from both phases provide the link between agricultural resource use and farm financial conditions, one of the cornerstones of the ARMS design. ERS and NASS jointly develop the questionnaires.

Manuals are produced each year to instruct enumerators and editors, and provide survey documentation. Both questionnaires and manuals can be found through visiting the ARMS questionnaires and manuals page.

Survey Scope

  • Time: Each year's survey asks for information from the prior year. For Phase II, the reference year corresponds to the crop's production cycle (from one harvest season to the next); for Phase III, the reference year is the calendar year (Jan. 1 to Dec. 31). ARMS has been administered every year since 1996, when it replaced the Farm Costs and Returns Survey.
  • Geography: ARMS collects information from farms in the 48 contiguous States. Some agriculturally important States are oversampled to provide enough sample coverage to allow representative selected State-level estimates. The larger samples were started in 2003 and data were first available in 2004. States that are oversampled are Arkansas, California, Florida, Georgia, Illinois, Indiana, Iowa, Kansas, Minnesota, Missouri, Nebraska, North Carolina, Texas, Washington, and Wisconsin. Other States are provided increased commodity specific samples to ensure crop or livestock coverage. See States surveyed by commodity and year below.
  • Farms: ARMS samples and collects information from farms of all sizes, family and nonfamily farms, and corporate and unincorporated farms. It does not collect information from prison farms or research farms. To receive the survey and have responses incorporated into the ARMS data, a place must qualify as a farm, which means a place from which $1,000 or more of agricultural products were produced and sold or normally would have been sold, during the year.
  • Households: ARMS only collects household information for the households of principal operators of family farms. The principal operator is the person who runs the farm, making the day-to-day management decisions. A family farm is where the principal operator and people related to the operator by blood, marriage, or adoption, have more than a 50-percent ownership interest in the farm.
  • Commodities: While farms growing all types of commodities are sampled every year, each year ARMS oversamples farms in one or more commodity specialization in order to produce Costs and Returns estimates. For sampled commodity crop producers, both a Phase II Production Practices and Costs Report and Phase III Costs and Returns Report are requested. For livestock commodities, only the farm-level Phase III questionnaire is necessary. The table below indicates which commodities were oversampled in each year since the inception of ARMS.
ARMS coverage by commodity and year
Commodity '96 '97 '98 '99 '00 '01 '02 '03 '04 '05 '06 '07 '08 '09 '10 '11 '12 '13 '14 '15 '16 '17 '18 '19 '20 '21 '22 '23*
Apples                       2,3                                
Corn 2,3 2 2 2 2 2,3       2,3         2,3           2,3         2,3    
Soybeans 2 2,3   2 2   2,3       2,3           2,3           2,3          2,3
Cotton 2 2,3   2 2     2,3       2,3               2,3       2,3        
Winter wheat 2 2 2,3 2 2       2,3         2,3               2,3         2,3  
Spring wheat 2 2 2,3   2       2,3         2,3               2,3         2,3  
Durum wheat 2 2 2,3   2       2,3         2,3               2,3         2,3  
Potatoes 2 2 2 2                                                
Rice         2,3           2,3             2,3               2,3    
Sorghum (milo)               2,3               2,3               2,3        
Tobacco 2,3                       3                              
Sugar beets         2,3                                              
Peanuts                 2,3                 2,3                    2,3
Sunflowers                                                        
Oats                   2,3                                    2,3
Barley               2,3               2,3               2,3        
Cow-calf 3                       3                   3          
Hogs     3           3         3           3         3      
Dairy         3         3         3           3         3    
Broilers                     3         3                        
Landlords (TOTAL)                                     4                  
2 = Phase II field-level Production Practices Report only.
2,3 = Both Phase II field-level Production Practices Report and Phase III whole-farm Costs of Production survey.
3 = Phase III whole-farm Costs of Production survey only.
4 = Tenure, Ownership, and Transition of Agricultural Land Survey (TOTAL).
*Planned.
Note: Years represent the reference period for the survey data, which is collected during the following year. Additional ARMS-based statistics about on-farm fertilizer and pesticide usage on selected field crops and potatoes are also collected and available through USDA, National Agricultural Statistics Services (NASS), Agricultural Chemical Use Program.		    

  • Commodities by State: The States included in commodity-specific surveys vary each year (depending on the crops surveyed) to help minimize respondent burden. The sampling used in ARMS Phase II was not intended to support State estimates, but sufficient observations are obtained in some States to allow reporting. However, the ability to partition data for individual States is very limited. For these estimates, States accounting for at least 90 percent of commodity production are sampled. The table below indicates which commodities were oversampled in each State, by year, since the inception of ARMS.
States surveyed by commodity and year
Apples
2007 CA MI NY NC OR PA WA                                                
Corn
1996     IL IN IA KS KY MI MN MO NE       NC   OH PA SC SD TX   WI                
1997     IL IN IA     MI MN MO NE           OH     SD     WI                
1998 CO   IL IN IA KS KY MI MN MO NE       NC   OH PA   SD TX   WI                
1999 CO   IL IN IA KS KY MI MN MO NE       NC   OH     SD TX   WI                
2000 CO   IL IN IA KS KY MI MN MO NE     NY NC ND OH PA   SD TX   WI                
2001 CO GA IL IN IA KS KY MI MN MO NE     NY NC ND OH PA   SD TX   WI                
2005 CO GA IL IN IA KS KY MI MN MO NE     NY NC ND OH PA   SD TX   WI                
2010 CO GA IL IN IA KS KY MI MN MO NE     NY NC ND OH PA   SD TX   WI                
2016 CO GA IL IN IA KS KY MI MN MO NE     NY NC ND OH PA   SD TX   WI                
2021 CO GA IL IN IA KS KY MI MN MO NE     NY NC ND OH PA   SD TX   WI                
Soybeans
1996 AR   IL IN IA     LA     MN     MS MO NE     OH       TN     WI          
1997 AR DE IL IN IA KS KY LA   MI MN     MS MO NE NC   OH PA SD   TN     WI          
1998 AR   IL IN IA KS KY LA   MI MN     MS MO NE NC   OH   SD   TN                
1999 AR   IL IN IA KS KY LA   MI MN     MS MO NE NC   OH PA SD   TN                
2000 AR   IL IN IA KS KY LA   MI MN     MS MO NE NC ND OH   SD   TN     WI          
2002 AR   IL IN IA KS KY LA MD MI MN     MS MO NE NC ND OH   SD   TN   VA WI          
2006 AR   IL IN IA KS KY LA   MI MN     MS MO NE NC ND OH   SD   TN   VA WI          
2012 AR   IL IN IA KS KY LA   MI MN     MS MO NE NC ND OH   SD   TN   VA WI          
2018 AR   IL IN IA KS KY LA   MI MN     MS MO NE NC ND OH   SD   TN   VA WI          
Cotton
1996   AZ AR CA GA LA MS         TN     TX                                
1997 AL AZ AR CA GA LA MS MO NC   SC TN     TX                                
1998 AL AZ AR CA GA LA MS   NC     TN     TX                                
1999 AL AZ AR CA GA LA MS   NC     TN     TX                                
2000 AL AZ AR CA GA LA MS MO NC     TN     TX                                
2003 AL AZ AR CA GA LA MS MO NC   SC TN     TX                                
2007 AL   AR CA GA LA MS MO NC   SC TN     TX                                
2015 AL AZ AR CA GA   MS MO NC   SC TN     TX                                
2019 AL AZ AR CA GA LA MS MO NC OK SC TN     TX                                
Durum wheat
1996       ND                                                      
1997       ND                                                      
1998 CA   MT ND SD                                                    
2000       ND                                                      
2004     MT ND                                                      
2009   ID MT ND SD                                                    
2017 CA ID MT ND   AZ                                                  
2022 CA ID MT ND   AZ                                                  
Other Spring wheat
1996     MN MT ND                                                    
1997     MN MT ND   SD                                                
1998   ID MN MT ND OR SD WA                                              
2000     MN MT ND   SD                                                
2004   ID MN MT ND OR SD WA                                              
2009 CO ID MN MT ND OR SD WA                                              
2017   ID MN MT ND OR SD WA                                              
2022   ID MN MT ND   SD WA                                              
Winter wheat
1996     CO DE   ID     KS                 MT NE         OK   OR   SD   TX WA
1997     CO     ID IL   KS               MO MT NE       OH OK   OR PA SD   TX WA
1998   CA CO   GA ID IL   KS   LA       MN MS MO MT NE   NC   OH OK   OR   SD   TX WA
1999               IN                                              
2000 AR   CO     ID IL   KS KY             MO MT NE   NC   OH OK   OR   SD   TX WA
2004     CO     ID IL   KS         MI     MO MT NE       OH OK   OR   SD   TX WA
2009     CO     ID IL   KS         MI MN   MO MT NE     ND  OH OK   OR   SD   TX WA
2017     CO     ID IL   KS         MI MN   MO MT NE     ND OH OK   OR   SD   TX WA
2022     CO     ID IL   KS  KY       MI     MO MT NE NM NC   OH OK   OR   SD TN TX WA
Peanuts
1999 AL   GA   NC TX                                                  
2004 AL FL GA   NC TX                                                  
2013 AL FL GA SC NC TX                                                  
Potatoes
1996 ID ME         WA                                                
1997 ID ME MN ND OR   WA WI                                              
1998           PA   WI                                              
Rice
2000 AR CA LA MS   TX                                                  
2006 AR CA LA MS MO TX                                                  
2013 AR CA LA MS MO TX                                                  
2021 AR CA LA MS MO TX                                                  
Sugar beets
2000 CA CO ID MI MN MT NE ND OR WA WY                                        
Sunflowers
1999 KS ND SD                                                        
Oats
2005 IL IA KS MI MN NE NY ND   PA SD     TX WI                                
2015 IL IA KS MI MN NE NY ND OH PA SD     TX WI                                
Barley
2003   CA   ID MN MT ND   PA SD UT       WA WI WY                            
2011 AZ CA CO ID MN MT ND OR PA         VA WA WI WY                            
2019 AZ CA CO ID MN MT ND OR PA  SD       VA WA WI WY                            
Sorghum
2003 CO KS MO NE   OK SD TX                                              
2011 CO KS   NE   OK SD TX                                              
2019 CO KS  MO NE NM OK SD TX                                              
Flue-cured tobacco
1996 GA NC SC VA                                                      
Tobacco
2008 GA KY NC SC TN VA                                                  
Cow-calf
1996     CA CO FL   ID IA IL KS KY     MO   MT NE NM ND OK OR   SD   TN TX     WY    
2008 AL AR CA CO FL GA   IA   KS KY     MO MS MT NE NM ND OK OR   SD   TN TX   VA WY    
2018 AL AR CA CO FL GA  ID IA IL  KS KY LS MN MO MS MT NE NM ND OK OR   SD   TN TX   VA WY    
Hogs
1998 AL AR CO GA IL IN IA KS KY MI MN     MO NE NC OH OK SC   SD   TN   UT VA   WI      
2004   AR CO GA IL IN IA KS KY MI MN     MO NE NC OH OK   PA SD         VA   WI      
2009   AR CO GA IL IN IA KS KY MI MN     MO NE NC OH OK   PA SD         VA   WI      
2015         IL IN IA KS   MI MN     MO NE NC OH OK   PA SD                    
Dairy
2000 AZ CA   FL GA ID IL IN IA   KY       MI MN MO NM NY OH     PA   TN TX   VT VA WA WI
2005 AZ CA   FL GA ID IL IN IA   KY       MI MN MO NM NY OH     PA   TN TX   VT VA WA WI
2010 AZ CA CO FL GA ID IL IN IA KS KY     ME MI MN MO NM NY OH OR   PA   TN TX   VT VA WA WI
2016 AZ CA CO FL GA ID IL IN IA KS KY     ME MI MN MO NM NY OH OR   PA SD TN TX UT VT VA WA WI
2021 AZ CA CO FL GA ID IL IN IA KS KY     ME MI MN MO NM NY OH OR   PA SD TN TX UT VT VA WA WI
Broilers
2006 AL AR CA DE GA KY LA MD MO MS NC     OK PA SC TN TX VA                        
2011 AL AR CA DE GA KY LA MD MO MS NC     OK PA SC TN TX VA                        
Tenure, ownership, and transition of Agricultural Land Survey (Total)
2014 AL AR CA FL GA ID IL IN IA KS KY     MI MN MS MO NE NC ND OH   OK   PA SD   TX WA WI  

Data Collection

ARMS data are collected by self-administered mailed questionnaires and personal interviews conducted by trained enumerators using several types of questionnaires. Mail responses can be returned on the printed form or through electronic data reporting (EDR). Data collection starts in June with Phase I, continues during the fall with Phase II (production practices and cost data), and finishes in late winter (after the close of the year) with Phase III (whole-farm income and costs data). Phase II data are collected at the individual field and production unit level, while Phase III is collected for the whole farm business. Survey instruments (available for download) show the range of questions asked for Phase II and Phase III on income, expenses, assets, debt, and specific crop/livestock management practices. Interviewers' manuals (also available for download) outline detailed enumeration procedures for each phase of the survey, providing specific instruction on how to conduct an interview and specific methods and examples on recording complicated data. Questionnaires include specific notes with each question to guide respondents.

Data Processing and Analysis

Data collection and preliminary editing/analysis and processing are managed by NASS Headquarters in Washington, D.C. and by 12 NASS regional offices throughout the contiguous U.S. About 200-300 NASS personnel as well as ERS personnel are involved in preliminary editing/analysis and processing prior to delivery of a raw survey file to ERS.

Flow chart shows the steps involved in collecting, processing, and delivering ARMS data used by researchers and policy analysts

Initial processing includes manual review and possible imputation by enumerators, supervisory enumerators, or regional statisticians for missing items or items that are not allowed to be initially coded as a nonresponse (-1). NASS then uses a set of automated processing tools, including FEITH, which provides scanned images of all questionnaires; PRISM2, an interactive data review system; and IDAS (Interactive Data Analysis System) for data review at the individual (respondent) and aggregated (strata, State, etc.) data levels.

Additional post-survey processing includes computer imputation to replace nonresponse codes for a number of items, survey weight calibration, outlier adjustment, and final summarization. ERS conditions the final raw survey data file with additional data editing, analysis, item imputation, and variable creation for use in research.

Editing

Survey enumerators, supervisory enumerators, and NASS regional office personnel provide a preliminary edit of completed ARMS questionnaires before entry into the NASS computer system. The questionnaires are then scanned into FEITH and data are manually keyed before being processed with a computerized edit program used to identify potential data errors. NASS staff in Washington and in regional offices, assisted by ERS staff, then check for errors and consistency using the PRISM2 editing program. After further analysis, NASS passes a completed raw survey file to ERS. ERS then creates a final dataset that includes some retained NASS variables, all keyed data, and several hundred additional calculated variables covering farm operation and farm operator household characteristics. ERS performs a final round of review, analysis, and updates of the complete ARMS research file prior to its release.

NASS Computer Edit/Analysis

The NASS computer edit identifies potential errors involving known physical relationships, obvious coding errors, and basic economic relationships between interrelated questionnaire items. Specifications for the computer edit are reviewed and updated annually by ERS and NASS. Errors/potential errors are divided into two categories—critical errors (the item value must be corrected or a comment explaining the value must be recorded), and warnings (the item value should be reviewed and/or updated).

NASS uses a combination of software tools including PRISM, FEITH, and IDAS to examine and evaluate ARMS data during the editing phase. IDAS is the principal tool for examination of individual report data. IDAS provides tabular and graphical displays of survey-level report items and additional analysis variables by State, region, and at the U.S. level. The objective is to facilitate the identification of remaining data errors. Custom Hyperion IR queries are also used if specific issues arise during analysis. Both ERS and NASS personnel participate in the editing and analysis of ARMS data with IDAS.

NASS Imputation

Survey responses are divided into two categories, those for which a value must be provided (either by the respondent or manually imputed by a NASS statistician) and those that can be initially coded as a nonresponse (the respondent refused to provide or did not know the answer). Items for which a value must be initially provided constitute a relatively small share (typically less than 30 percent) of all items. Statisticians are instructed to manually impute values for these items using one or more methods/sources including relationships in the questionnaire, similar reports, other State surveys, other outside sources and publications, and data from the ERS website.

Most of the items on the ARMS Phase III survey can be initially coded as a nonresponse (the respondent did not know or refused to provide an answer to an item). In 2005, for example, about 75 percent of all items fell in this category. At the request of ERS, NASS uses a computer algorithm to impute data for a small subset of these items (144 in 2011)—typically, about 10 percent of all items that can be coded as nonresponses. The items selected by ERS are based on its mission requirements for time sensitive development of farm operation/farm operator household financial and structural characteristics and by NASS’s need to complete analysis and the Farm Production Expenditures Summary publication. The remaining missing items are coded with a value of -1 in the file delivered to ERS.

Beginning in 2014, item level imputations are now done using a multivariate approach. Prior to the implementation of the multivariate approach, NASS used an un-weighted conditional means imputation system that placed records into homogenous groups and imputed based off of reported data from those groups. The new multivariate approach uses a regression-based technique that allows for flexibility in the selection of conditional models while providing a valid joint distribution. In this procedure, labeled as Iterative Sequential Regression (ISR), parameter estimates and imputations are obtained using a Markov chain Monte Carlo sampling method. Using ISR, we are better able to preserve the relationships within the data and also allow the imputed values to better represent the variability of the data. Records with imputed data are re-edited to ensure the returned value is acceptable. The imputation algorithm delivers an acceptable value more than 99.5 percent of the time and Field Office statisticians are required to manually impute for any missing items.

Before 2014, the NASS imputation algorithm computed unweighted mean values for donors (farms reporting a positive value for an item) in a specific set of farm groupings based on location, farm size, and farm type. Donors with "extreme values" are excluded from calculations. In assigning an imputed value, the algorithm uses the first grouping (starting with group 1) that contains at least 10 donors. In 2012, 80 percent of imputed values were obtained from group 1. Means are computed for 12 imputation groups:

  1. Region, sales class, farm type
  2. Region, pooled sales class, farm type
  3. U.S., sales class, farm type
  4. U.S., pooled sales class, farm type
  5. Region, sales class, pooled farm type
  6. Region, pooled sales class, pooled farm type
  7. U.S., sales class, pooled farm type
  8. U.S., pooled sales class, pooled farm type
  9. Region, farm type
  10. Region, sales class
  11. Region average
  12. U.S. average

Farm depreciation expense, landlord taxes, and other assets are examples of individual items most frequently imputed. The remaining items for which data are most commonly imputed can be grouped into three categories: farm labor, other farm-related income, and farm assets.

Calibration

After editing and imputation, survey sample weights are calibrated so that weighted survey totals for selected items match official USDA estimates for production or acreage where possible. A decision rule is employed to exclude calibration targets for survey items whose weighted totals fall below or above predetermined thresholds in relation to official estimates. Targets are added in the year when commodities of interest are over-sampled if they are not already part of the targets listed below.

The calibration items in the last few years have numbered between 30-32 targets as follows:

  • Total number of farms
  • Number of farms by economic sales class (8 groups):
    • Less Than $10,000
    • $10,000 - $49,999
    • $50,000 – $99,999
    • $100,000 – $249,999
    • $250,000 – $499,999
    • $500,000 – $999,999
    • $1,000,000 – $4,999,999
    • $5,000,000 and Over
  • Physical production of (e.g. corn, harvested acres; hogs, number of head):
    • Broilers
    • Cattle
    • Cattle on feed
    • Corn
    • Cotton
    • Eggs
    • Fruits
    • Hay
    • Hogs
    • Milk
    • Peanuts
    • Soybeans
    • Tobacco
    • Turkeys
    • Veggies
    • Sugar
    • Wheat
    • Oats
    • Sorghum
    • Barley
    • Rice
    • Beef
    • Organic milk
    • Millet
    • Sunflowers
    • Apples
    • Potatoes
    • Nursery/Greenhouse in the open
    • Nursery/Greenhouse under glass or other protection

Outlier Process

Following calibration, outliers are identified and reviewed by the official USDA-NASS National Outlier Review Board. There are typically five or fewer national outliers each year. An outlier is defined as a sampled farm where weighted (expanded) data for total expenses accounts for 0.5 percent of U.S. total expenses and/or 2.5 percent of regional total expenses. The review board usually adjusts outlier sample weights downward. The general rule for treating outliers is to reweight to the median weight of the matching economic class by farm type. Following the National Outlier Review Board, outlier weights are frozen at board levels and the calibrated summary is rerun. Following recalibration, survey weights are rechecked for the possible introduction of new outliers. An iterative procedure is used to adjust the weights of the new outliers, and recalibrate the weights until outliers are mitigated.

ERS Dataset Conditioning

Following receipt of the raw survey file from NASS, ERS creates a preliminary version of the final ERS research file. The majority of NASS calculated variables are removed mainly because they cannot be updated if changes are made. Conditioning programs calculate several hundred chronologically and methodologically consistent financial and operational variables and add them to the raw survey data file. The additional research variables are typically calculated from a combination of raw survey items. ERS adds geographic identifiers (such as ERS farm resource regions), demographic information (for example, categorical variables based on the U.S. 2010 Population Census), and other county, regional, and national economic information. ARMS III data files are available for 1996 to present.

ERS Imputation

In creating the final research file, ERS imputes data for about 40 items that are necessary to meet time-sensitive ERS mission requirements for the publication of farm operation and farm operator household data. This includes farm operator debt and items pertaining to the farm operator and the farm operator’s household. Collectively, ERS and NASS impute data for about 15 percent of all items that could be coded as nonresponses. The remaining items are retained with a "-1" code in the final ERS research file.

In general, ERS employs the same basic methodology as NASS, using mean values for responding farms to replace refusal codes. ERS imputation schemes can be procedurally complex incorporating other survey items and varying in some cases by questionnaire version and the level of item aggregation. ERS imputes values for a number of items related to the farm operator and operator’s household, including:

  • Off-farm income items—wages and salaries, net income from other farms, net income from other business, net income from renting farm land, interest income, dividend income, capital gains/loss, private retirement income, public retirement income, and other
  • Share of farm income retained by operator household
  • Farm operation debt
  • Operator household size
  • Household expenses, nonfarm assets, and nonfarm debt
  • Total value farm sales, net operating income, and total off-farm income previous year.

ERS uses a larger set of classification variables (in addition to farm size, type, and location) in imputing household items, including the number of operators, the operators’ age class, education level, marital status, and retirement status, as well as the farms' legal organization. Differences in questionnaire versions are also incorporated into imputation methods for the farm operator’s household.

ERS Data Review

Following creation of the preliminary version of the research data file, ERS performs a final review of the data. Key financial and operational values and relationships are audited for internal (within record) and external (across records and across time) consistency. SAS programs are used to identify errors in the computer code used to create the final research file and remaining problems in the data that were not identified during the NASS edit/analysis. Some records are updated by the conditioning programs while others are researched again at NASS and updates applied if errors are found. In evaluating potential data edits ERS also uses the auxiliary comment file provided by NASS. The comment file includes explanations at the survey record (farm) level by ERS and NASS record analysts about enumerator comments or "unusual data values" remaining in the survey file.

Quality Control

ERS and NASS provide "train the trainer" workshops prior to data collection for each survey. Regional and State statisticians then train enumerators through a series of dispersed workshops. ERS and NASS develop and provide training materials to the State survey statisticians. After questionnaires are completed by the enumerators, each questionnaire is reviewed by supervisory enumerators for completeness, inconsistent responses, or errors, and then transferred to a NASS State or Regional office where each questionnaire is reviewed before it is keyed into an electronic format. While the survey database is assembled, a computer edit is used to identify potential recording errors or inconsistencies in data relationships (for example, interest expense matched with farm debt). In the process of conditioning the survey database, uncharacteristic responses are investigated and data are verified or corrected. Select cells can be marked for later computer imputation. The Interviewer’s manual and the Survey Administration manual are updated each year and provide specific details to provide uniform treatment of data collection and data processing procedures.

Changes in methodology
Time period Change
2013 The survey was redesigned to reduce expenditures while maintaining sample size. The Core version of ARMS Phase III was eliminated, and questionnaire length in other Phase III versions was reduced. Remaining versions shifted to mail mode, while retaining enumerator assistance on request.
2012 The ARMS questionnaire was integrated with the Census of Agriculture; ARMS questionnaires included all census questions, and primary ARMS questions.
2010–present New top-end value codes were introduced for other farm assets and off-farm income, assets, and debt, replacing the top code of 1.5 million and above that had been in place since 1996. The new top code is 10 million and above, and 4 additional codes were introduced to cover values between 1.5 million and 9.99 million.
2007 The ARMS questionnaire was integrated with the Census of Agriculture; ARMS questionnaires included all census questions, and primary ARMS questions.
2003–2012 A new, shorter, core version of the survey was introduced, as a mail-out survey with enumerator follow-up upon request. The added sample size allowed for State-level estimates to be generated for 16 major agricultural States. The number of useable responses rose to 20,000 or more after core introduction, compared to 10,000 before.
2002 The ARMS questionnaire was integrated with the Census of Agriculture; ARMS questionnaires included most census questions, and primary ARMS questions.

Variance Estimation

ERS and NASS follow strict statistical procedures when designing, collecting, and summarizing ARMS data. The observations from any given sample of the population must be weighted properly to ensure that sample estimates of totals and averages are representative of the total farm population. Through a process known as calibration survey weights are adjusted so that sample estimates of population totals hit known targets such as the total acres of corn.

Estimating variances of sample estimates gives an indication of how an estimate may vary if a different ARMS sample were drawn. An estimate’s variance gives a sense of how close a sample estimate is likely to be to the true population value. Given the complex design of ARMS, a specific method for estimating variances, known as the Delete-a-Group Jackknife, is recommended for calculating the variance of a sample estimate.

The following documents include information on the calibration of sample weights and variance estimation.

  • An Overview of Calibration Weighting and the Delete-a-Group Jackknife. The document details the statistical theory behind the use of calibration and the variance estimation technique called the "delete-a-group jackknife." Contact the ERS ARMS Team for a copy of this paper.
  • Variance Estimation with USDA’s Farm Costs and Returns Surveys and Agricultural Resource Management Study Surveys. This paper is an overview of survey estimators, sample design, hypothesis testing, disclosure rules, and reliability measures for ARMS followed by statistical program documentation. Sums, ratios, means, multiple regression, binomial logit analysis, and order statistics are covered. Contact the ERS ARMS Team for a copy of this paper.

Nonresponse

As with most surveys, some sampled farm operators do not respond to particular questions (item nonresponse) or to the entire survey (unit nonresponse). For some questions, item nonresponse is addressed by imputation, which is a statistical procedure that uses information from other questions to fill in missing values. Unit nonresponse, in contrast, is at least partially addressed through adjustments to sample weights, including calibration, that help to account for the missing farms.

The following publications provide information such as patterns in nonresponse and potential consequences and solutions.

Technical Documents

Several documents are available which explain in technical detail various issues related to the ARMS.

Maps

This set of maps helps data users visualize and better understand the geographical scope and level of aggregation by which many ARMS data are summarized. For example, the ARMS web data tool presents farm financial estimates aggregated to Farm Resource Regions, which do not follow State boundaries. Maps below demonstrate the different ways data are often summarized.

U.S. Farm Resource Regions: Farm resource regions are defined using farm production regions, land resource regions, crop reporting districts, and farm characteristics. The regions are designated at the county level and therefore do not generally follow State boundaries. See the ERS report, Farm Resource Regions, (AIB-760, August 2000), for more information, or download the county-to-ERS Resource Region aggregation in Excel.

A map of U.S. Farm Resource Regions

Farm Production Regions: The older Farm Production Regions, in following State boundaries, group unlike areas together because a single State often encompasses different soils and typography. For example, the old Appalachian Region, comprised of Tennessee, Kentucky, North Carolina, West Virginia, and Virginia, contains the Appalachian Mountains, Piedmont, and Coastal Plain areas, all of which have quite different agriculture.

A map of Farm Production Regions

Patterns of Agricultural Diversity: County clusters, based on types of commodities produced, have shown that a few commodities tend to dominate farm production in specific geographic areas that cut across State boundaries. The climate, soil, water, and topography in localized geographic areas tend to constrain the types of crops and livestock that will thrive there. See Diversity in U.S. Agriculture: A New Delineation by Farming Characteristics, AER-646, July 1991, for more information.

USDA Land Resource Regions: In constructing the ERS production regions, analysts identified where areas with similar types of farms intersected with areas of similar physiographic, soil, and climatic traits, as reflected in USDA's Land Resource Regions.

A map of USDA Land Resource Regions

NASS Crop Reporting Districts: County reporting districts influenced the construction of the ERS farm resource regions by conforming intersecting areas to follow the boundaries of NASS Crop Reporting Districts (CRD), which are aggregates of counties. With more and more data available at the county level, geographic representations need no longer be constrained to follow State boundaries.

A map of NASS Crop Reporting Districts

NASS ARMS Regions: Five Farm Production Expenditure Regions

Other Resources

Recommended Citation

U.S. Department of Agriculture, Economic Research Service. ARMS Farm Financial and Crop Production Practices.