How do we make emission factors in purchaser value?

[WesIngwersen]

Emission factors can be prepared in purchaser value, which includes the value of wholesale/retail and distribution (aka margins in IO language). Factors in this valuation are more appropriate for users that want to multiply spend data by emission factors to get final emissions.

In USEEIO, emission factors are first computed using the model's valuation, which is in producer price, and the models scope, which as defined by a producer price model is "cradle to gate". The PRO factors are converted into PUR factorse using commodity-specific margin rations of $ margin per $ PRO, which gives the Supply Chain Emission Factors (SEF). Also the impact of margins is calculated based on the wholesale, retail and distribution portions of that margin but specific margin commodities within that are not used. These are estimated in PUR price and called "Margin Emission Factors" (MEFs). These are added together to get the final factors which represent a "Cradle-to-shelf" for commodities purchased at a retail store, or in the case of items shipped from a retailer to the final user, "Cradle-to-user".

$SEF (PUR) + MEF (PUR) = SEF with margins (PUR)$

[WesIngwersen]

Even assuming we start with a PRO model the data requirements for this transformation suggest that it be included in milestone 1

[WesIngwersen]

Adding more clarity to the USEEIO approach, there are two steps:

1. The PRO -> PUR price transformation
   In USEEIO, the Detailed Margins table, which only exists for benchmark years, is used as the first source. The table includes the commodity, the purchaser, the producer value of the commodity, a wholesale, a retail, and a transportation value for every non-zero transaction in the Use table. Records for margins for export, import and change in inventories are removed, some commodities are remove and other adjustments are made. This is done in getMarginsTable()
   https://github.com/cornerstone-data/useeior/blob/90f4d7a022a3ecb2043535c30b473cd1068078a8/R/LoadMargins.R#L6-L67

The margins table is then converted into the Phi matrix in calculateProducerbyPurchaserPriceRatio()
https://github.com/cornerstone-data/useeior/blob/90f4d7a022a3ecb2043535c30b473cd1068078a8/R/AdjustPrice.R#L47-L66
In this Phi preparation, inflation/price years adjustments are made to the commodity producer price based on the commodity Rho value and to the margins sectors based on a margins sector Rho value.

Thus the application of Phi ratios used for this conversion are both commodity and year-specific, although the ratios themselves are based on those in the benchmark year.

2. The estimate of the impacts of the margins
   This is done in useeior in calculateMarginSectorImpacts()
   https://github.com/cornerstone-data/useeior/blob/90f4d7a022a3ecb2043535c30b473cd1068078a8/R/CalculationFunctions.R#L487-L524
   Margins coefficients which are margin:PRO ratio are calculated for each commodity.
   For each type of margin, e.g. transportation, this ratio is allocated to specific transportation sectors based on their total commodity output (acknowledging that transportation type is not commodity-specific)
   Those quantities representing inputs of margins put commodity are put into an A matrix like format (commodity x commodity) where the rows are the margin sectors and columns are commodities using them) and that gets multiplied by existing M and N matrices. Note the resulting matrices called "M_margin" and "N_margin" are still in PRO value (which is counter-intuitive).

Note that in application like here, then both the values taken from N and N_margin are price adjusted using Phi, and then can be reported together where they represent the SEF and MEF, respectively, in PUR value.
Example of application is here in the Supply Chain factors preparation:
https://github.com/cornerstone-data/supply-chain-factors/blob/1ecb5ff6a2464d52ce0b37a5f1c859a25b0cb069/CalculateEmissionFactors.Rmd#L20-L38

[MoLi7]

In CEDA, emission factors (EFs) are generated by default in producer price, too. The PRO EFs are converted to PUR EFs by multiplying with the producer-to-purchase ratios derived from the commodity-specific Margins table by BEA.

[WesIngwersen]

Clarifying after discussion with @MoLi7 , that in CEDA, there is:

1. Option PRO -> PUR price transformation, filtering out of final uses and post-processing to replace values where <0 or >1 with the average of that range.

2. There is no equivalent estimation of the environment impact of the margins, due to uncertainty in the data.

[WesIngwersen]

Note the CEDA approach summary was revised at time of this comment

[WesIngwersen]

Filtering of margins data in the PRO:PUR ratio calculation. This was not described well above. Neither model used the Margins data in full, as is.

1. USEEIO. Filtered out records for the bookkeeping commodities + scrap and used goods. S00401 Scrap, S00402 Used and secondhand goods, S00300 Noncomparable imports, S00900 Rest of the world adjustment. Then it filtered our purchases related to Import, Export, and Change in Inventories. F04000, F05000, F03000. And if any remained Producer Values in the table were negative, then those records were flipped to positive.
2. CEDA. Filtered out all records for final uses F*.

[WesIngwersen]

Thoughts on filtering. Any filtering of records will result in values that are inconsistent with the total commodity output in the PUR price Use table. But we don't use that table as a benchmark. We just use the data to construct an average Producer:Purchaser Price ratio, and also to estimate emissions associated with those margins.

FOR REFERENCE THE RAW MARGINS FILE WE USE IS HERE

Implications of filters by type: purchaser filters

1. Final uses. This precludes margins of purchases for households, investment purposes, government uses, and international trade, and change in inventories.
   The PCE (household) F01000, Residential Investment F02R00 and all the government fina uses are not private industry . They do generally have significantly higher relative margins especially for household purchases.
   Imports F05000 have no margins in the Margins table.
   Exports F04000 do have margins, but the final consumers for these are out of the country.
   Change in inventories F03000 have margins - see first 8 rows

Commodity Description	Producers' Value	Transportation Costs	Wholesale	Retail	Purchasers' Value
Oilseed farming	-1330	71	227	0	-1032
Grain farming	-2056	172	337	0	-1547
Vegetable and melon farming	-673	25	64	0	-585
Fruit and tree nut farming	-758	25	57	0	-676
Greenhouse, nursery, and floriculture production	84	11	23	0	119
Other crop farming	-823	20	49	0	-754
Dairy cattle and milk production	100	12	11	0	122
Beef cattle ranching and farming, including feedlots and dual-purpose ranching and farming	1101	68	45	0	1214

Note that some of the PRO value is negative, because this means that the item is actually consumed from inventory and not added to inventory. Using the negative PRO items directly in the PRO:PUR formula is problematic because you get ratios that are potentially negative or >1. Real margins mean this ratio should always be < 1 because it there is positive value in wholesale,retail and transportation. That margin value is occuring in the current year whereas when PRO values are negative in this column it means they were produced in a previous year and were in stock.

Change in inventories is further complicated because in some cases the margin sector values are negative as well. Indicating that this margin activity happened prior to 2017.

The nonresidential private fixed investment uses F02E00, F02N00 and F02S00 include (and is dominated by) private industry. In fact, for many commodity types like Automobiles, this is the place where industry purchases show up in the Use table, e.g. here are the only items for the Automobile commodity in the 2017 Margins.

Industry Code	Industry Description	Commodity Code	Commodity Description	Producers' Value	Transportation Costs	Wholesale	Retail	Purchasers' Value
336111	Automobile manufacturing	336111	Automobile manufacturing	14	0	0	0	14
F01000	Personal consumption expenditures	336111	Automobile manufacturing	20107	967	6426	42401	69901
F02E00	Nonresidential private fixed investment in equipment	336111	Automobile manufacturing	58608	2869	19336	3146	83960
F03000	Change in private inventories	336111	Automobile manufacturing	3227	153	1015	0	4395
F04000	Exports of goods and services	336111	Automobile manufacturing	17269	823	5445	0	23537
F05000	Imports of goods and services	336111	Automobile manufacturing	-66987	0	0	0	-66987
F06E00	Federal national defense: Gross investment in equipment	336111	Automobile manufacturing	29	1	9	0	39
F07E00	Federal nondefense: Gross investment in equipment	336111	Automobile manufacturing	77	4	25	0	105
F10E00	State and local: Gross investment in equipment	336111	Automobile manufacturing	436	21	140	0	597

Note here an example of where the Retail margin is relatively MUCH HIGHER for PCE/households than it is for the other purchasers.

[WesIngwersen]

Implications of filters by type: commodity filters

The Cornerstone model will remove the bookkeeping commodities and scrap S00401 (#5) so this removal would happen anyway.

S00402 Used and Secondhand goods is one remaining in the Cornerstone model that was filtered by USEEIO. USEEIO filtered it because of the negative producer values, representing donation of these goods, due to the math problems stated above.
In 2017 the total producer value of S00402 for many records is negative as is the purchasers value, which makes the current math invalid as well.

[WesIngwersen]

The margins sectors themselves (42*) sectors are also worth pointing out. They were not explicitly filtered by CEDA and USEEIO (although USEEIO is somewhere setting to 1). but the Margins data actually has zero or very small PUR value, so the PRO:PUR are either NAN (if divide by zero) or very high if it is constructed with them in place.

[WesIngwersen]

My recommendations:

If we assume Cornerstone EFs are intended for:

1. Industry customers
2. Incorporate the average margin value for all commodity for purchases for a given year, no matter when the commodity was made (take into account a durable good that comes from inventory)
3. Assume the purchase is for consumption in the U.S.

then my recommendation for margin treatment is the following:

1. Solve the math problem with negative values by using absolute values. Margins are always proportional to magnitude of the values, not the sign. Absolute value will solve this. USEEIO only applied this to Producer Price. For some of the change in inventories items the margins sector inputs are also negative; this can be extended to be applied to the margin sector inputs as well. Since we are only using the table here for developing these ratios and not for totals, this is acceptable.
2. Remove final uses by households, government, export and import. This leaves the change in inventories, and nonresidential investment categories.
3. Remove special commodities except Used and secondhand goods. The first treatment will solve the math problem
4. Remove the margins commodities. They should be treated as not having margins, so the PRO:PUR ratios should be 1.
5. Remove government industry uses (G*). Government appears in these two places in the Use table (as industry and as a final user). To be consistent with assumption 1 above, remove both.
6. Set all remaining values to 1. Many of the commodities that don't have margins do not mean they lack data - this means they have no margin as it is defined. Setting to 1 means PRO price = PUR price for these commodities.
7. Inflate margins to model_base_year USD. To be consistent with treatment of other matrices like A and B, the margins data also needs to be inflated. I recommend using applying commodity prices indices following with the USEEIO approach to inflate the margins data before creating the final ratios.

[WesIngwersen]

Note just added recommendation 4.

[bl-young]

I think what you described is right. It sounds like after group discussion that there could be benefit to providing margins for different groups of users. Margins that only account for industry-to-industry transactions, and separate margins for households or government - so keeping the code flexible for now will be useful. We decided that industry-specific margins is likely too much detail and not likely to be utilized.

There also was discussion of providing the margins by transport, wholesale, and retail separately to facilitate advanced users who have additional information about their purchases which might require removing some components.

[WesIngwersen]

Note just added recommendation 5 and 6.

[WesIngwersen]

Note just added recommendation 7.