Skip to content

update recursive-refac with sector_mapping (includes up-to-date develop branch) - #330

Merged
catherinebirney merged 658 commits into
recursive-refacfrom
sector_mapping
Apr 12, 2023
Merged

update recursive-refac with sector_mapping (includes up-to-date develop branch)#330
catherinebirney merged 658 commits into
recursive-refacfrom
sector_mapping

Conversation

@catherinebirney

@catherinebirney catherinebirney commented Mar 8, 2023

Copy link
Copy Markdown
Contributor

relevant to recursive-refac:

  • sector hierarchy updates: 'aggregated' or 'disaggregated' ->' flat' or 'parent-completeChild' or 'parent-incompleteChild'
  • update map_to_sectors() to map differently depending on sector hierarchy
  • new functions:
    -- return_primary_activity_column()
    -- estimate_suppressed_sectors_equal_attribution()
  • updates to water_national_2015_m1 FBS to recursive method

relevant updates from master:

  • updates to GHG FBS
  • updates to github actions
  • new gov and household sector codes
  • new 7-digit waste management pathways
  • sankey diagrams
  • standard graph colors based on sectors
  • new FBA/FBS
  • new attribution sources col
  • material crosswalk mapping
  • begin to convert to use of pathlib Path

catherinebirney and others added 30 commits December 1, 2022 09:48
@catherinebirney
set kaleido requirements based on os
e0aadb4
@catherinebirney
do not need to explicitly import kaleido as plotly automatically uses…
eb46af1 
… kaleido to write static images if kaleido installed
@catherinebirney
update kaleido use explanation
ff52189
@catherinebirney
Merge pull request #300 from USEPA/develop
7f275b4 
update v1.3-release with develop
@catherinebirney
update package version and drop develop from package requirements
fb50ef6
@catherinebirney
option to specify name of the FlowAmount col in aggregator()
5237d68
@catherinebirney
in compare_fbs_results() aggregate dfs before merge to drop duplicate…
393c7fc 
… rows caused by dropping metasources/attributionsources cols
@bl-young
Merge branch 'develop' into ghgi
aec3299
@bl-young
update GHG methods with new FBA data
d6615ac
@bl-young
initiate replacement of appendix tables with 3-13 for transport
b3842f6
@catherinebirney
Merge branch 'develop' into DataVis
0d7bec0
@catherinebirney
option to pass filename in sankey and stackedbarchart functions
1dc16ad
@catherinebirney
Merge pull request #270 from USEPA/DataVis
6183153 
Data vis
@catherinebirney
Merge branch 'develop' into v1.3-release
509762e 
# Conflicts:
#	requirements.txt
#	setup.py
@catherinebirney
add AttributionSources to FBS spec
f303e33
@catherinebirney
clarify loading yaml, not the parquet
32ed6a7
@catherinebirney
Merge pull request #301 from USEPA/v1.3-release
41206c8 
V1.3 release
@catherinebirney
add develop back to package requirements
c769ae1
@bl-young
add additional flow updates; align 2017 FBS with 2016
3b2a983
@bl-young
fix GIT_HASH import error
554a0cb
@bl-young
Merge branch 'develop' into ghgi
cbd1d07
@bl-young
continue updating ghgi fbs methods with new FBA tables
a6e0e64
@bl-young
add GHG 2020
e25088c
@bl-young
remove unused table A-4
a052937
@bl-young
fix parsing error in A-97
e03aa48
@bl-young
fix error in chapter name
5ed5865
@bl-young
temporarily suspend replacing appendix table with 3-13
094a4ee
@bl-young
amend 094a4ee
ada958b
@bl-young
reversion of 61c8e38, except use version('flowsa') as backup
b838300
@bl-young
fix GIT_HASH import error
4579b74
catherinebirney and others added 26 commits April 6, 2023 15:42
@catherinebirney
add method of replacing nonexistent state values with national data
2e334d2
@catherinebirney
new substitute_nonexistent_values() as method to add all possible opt…
8d8671d 
…ions for location/activity Budugs
@catherinebirney
skip validation that df values equal original values if multiplation,…
47e47b4 
… weighted average, or adding new data
@catherinebirney
warn if any activities are not mapped to sectors
c839bb5
@catherinebirney
add data_format for food waste method datasets
ea0e944
@catherinebirney
convert thousand USD to USD
ca3e032
@catherinebirney
convert foodwaste_use() to recursive method
0c7ca26
@catherinebirney
convert food_waste_national_2018_m1 to recursive method - total value…
d3b4cb2 
…s do not fully match original method
@bl-young
remove deprecated files
e75c8a1
@bl-young
reorder sources
83cf45c
@bl-young
Merge pull request #332 from USEPA/ghgi_v2
8edd34e 
Update GHGI method to v2.0
@catherinebirney
drop yamls that reference additional FBAs used in FBS - those FBAs no…
9ea90f9 
…w included in FBS yaml
@catherinebirney
add data formats
f969a09
@catherinebirney
comment out non-recursive function
82d36e3
@bl-young @catherinebirney
initial implementation of exclusion_fields
268acc9
@bl-young
initial update to write metadata, add v and githash to filenames
961926c
@bl-young
add AttributionSources field
606f202
@catherinebirney
move weighed_average() and substitute_nonexistent_values() to new flo…
94cc2b8 
…wsbyclean.py
@catherinebirney
new '!clean_function:' to call on functions in flowbyclean.py
1ef9e48
@bl-young
avoid KeyError for sources with no activity sets
25cc9c1
@catherinebirney
drop yamls that reference additional FBAs used in FBS - those FBAs no…
eb9c144 
…w included in FBS yaml
@catherinebirney
address year error in metadata
89ff261
@catherinebirney
metadata - drop call to old FBA yaml that is no longer used and updat…
96516db 
…e "allocation" to "attribution"
@bl-young
Merge pull request #337 from USEPA/metadata
a5ea5f8 
Update saving of file and metadata
@catherinebirney
add data_format to source_catalog
410508f
@catherinebirney
initial efforts to transition food waste m2 to recursive method
144ff6e
matthewlchambers
matthewlchambers approved these changes Apr 12, 2023
View reviewed changes

@matthewlchambers matthewlchambers left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should all be basically fine, noting that I haven't tested all the additions to flowby.py. Also, in the recursive branch I don't think flowbysector.py is getting used much if at all, so I hope that the changes to flowbysector.py are accomplishing their intended goal.

@catherinebirney

catherinebirney commented Apr 12, 2023

Copy link
Copy Markdown
Contributor Author

I think this should all be basically fine, noting that I haven't tested all the additions to flowby.py. Also, in the recursive branch I don't think flowbysector.py is getting used much if at all, so I hope that the changes to flowbysector.py are accomplishing their intended goal.

@matthewlchambers I updated sector_mapping with the most recent changes to develop/master branches, so the changes in flowbysector.py are not applicable to the recursive code.

Thanks for the review - I will go ahead and merge

@catherinebirney
catherinebirney merged commit 92c20e9 into recursive-refac Apr 12, 2023
@catherinebirney
catherinebirney deleted the sector_mapping branch April 12, 2023 19:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer
@matthewlchambers matthewlchambers matthewlchambers approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@catherinebirney @WesIngwersen @bl-young @matthewlchambers