****** PLEASE NOTE: These files were updated on 8/15/2011. Please reread the technical descriptions below if you used an earlier version as the data is now packaged in a more convenient format. Also note that "TNIC-3" industries were previously referred to as "VIC-7.06 industries". The TNIC-3 industries provided here are improved versions of the same classifications. ****************************** This file accompanies the TNIC-3 industry concentration database and describes where the data comes from, the papers that should be cited when providing academic references, and some very important technical details regarding its usage. Please read the technical details in full before using this data. These details are critically important to ensure proper usage. The data is at the firm-year level. ********************************************************************************************************************************** ********************************************************************************************************************************** ******************************************* General Background on TNIC industries ************************************************* ******************************************* General Background on TNIC industries ************************************************* ******************************************* General Background on TNIC industries ************************************************* ********************************************************************************************************************************** ********************************************************************************************************************************** For an extensive description of this data, please read the data and methodology sections of the studies noted below. Here is a brief description. This data is based on web crawling and text parsing algorithms that process the text in the business descriptions of 10-K annual filings on the SEC Edgar website from 1996 to 2008. These product descriptions are legally required to be accurate, as Item 101 of Regulation S-K legally requires that firms describe the significant products they offer to the market, and these descriptions must also be updated and representative of the current fiscal year of the 10-K. We merge each firm's text product description to the CRSP/COMPUSTAT universe using the central index key (CIK) [We thank the Wharton Research Data Service (WRDS) for providing us with an expanded historical mapping of SEC CIK to COMPUSTAT gvkey, as the base CIK variable in COMPUSTAT only contains current links]. Our resulting database is based on all publicly traded firms (domestic firms traded on either NYSE, AMEX, or NASDAQ) for which we have COMPUSTAT and CRSP data. We calculate our firm-by-firm pairwise similarity scores by parsing the product descriptions from the firm 10Ks and forming word vectors for each firm to compute continuous measures of product similarity for every pair of firms in our sample in each year (a pairwise similarity matrix). This is done using the cosine similarity method, which is applied after basic screens to eliminate common words are applied (see studies noted below). For any two firms i and j, we thus have a product similarity, which is a real number in the interval [0,1] describing how similar the words used by firms i and j are. The TNIC-3 classification data we are distributing only records firms having pairwise similarities with a given firm i that are above a threshold as required based on the coraseness of the three digit SIC classification. The level of coarseness of TNIC-3 thus matches that of three digit SIC codes, as both classifications result in the same number of firm pairs being deemed related. For example, if one picks two firms at random from the CRSP/COMPUSTAT universe, the likelihood of them being in the same three digit SIC code is 2.05%. Analgously, when the TNIC-3 cutoff is specified using our approach, the likelihood of two randomly drawn firms being deemed related in their TNIC-3 is also 2.05%. Hence, TNIC-3 is constructed to be "as coarse" as are three digit SIC codes. Note: TNIC industries are also purged for vertical relationships from the input/output tables (see paper for details). Note 2: The words used to construct TNIC industries only include nouns or proper nouns (see paper for details) and we exclude geographic terms. ************************************************************************************************************** ************************************************************************************************************** ********************************************** Citations ***************************************************** ********************************************** Citations ***************************************************** ********************************************** Citations ***************************************************** ************************************************************************************************************** ************************************************************************************************************** Please cite the following study when using this HHI data: Text-Based Network Industries and Endogenous Product Differentiation Gerard Hoberg and Gordon Phillips, University of Maryland Working Paper. * If using TNIC data beyond HHIs, please also cite the RFS study referred to in the readme file associated with TNIC industry classification data. ********************************************************************************************************************** ********************************************************************************************************************** ********************************************** Technical Details ***************************************************** ********************************************** Technical Details ***************************************************** ********************************************** Technical Details ***************************************************** ********************************************************************************************************************** ********************************************************************************************************************** Please read the following carefully to ensure proper usage of this data. Technical Note 1) Each file contains a gvkey and year firm identifier. The TNIC3HHI variable is the concentration measure and TNIC3TSIMM is the total similarity measure. Each observation should be mapped to COMPUSTAT using fiscal year endings that match the year field in this data. It is important to note that we already did the merge to COMPUSTAT, so you do not have to repeat this, which is why we provide data with gvkey as the identifier. The data contained here is not lagged. Consider a COMPUSTAT firm with a fiscal year ending on Sept 30th, 1997, for example. The corresponding records for this firm's gvkey in 1997 are based on the product description of the 10-K report that was associated with this firm's 9/30/1997 fiscal year end. Researchers needing lagged data must lag the data on their own. Technical Note 2) These HHI data are computed using TNIC designations that include the firm itself in part of the HHI calculation. All HHIs are based on firm sales data from COMPUSTAT, and are computed using the Herfindahl-Hirschmann sum of squared market shares formulation.