Skip to content

Set up a cleanup cron job that checks all the entries in the database for existance #1467

New issue
New issue
Closed
Closed
Set up a cleanup cron job that checks all the entries in the database for existance#1467
Assignees
andrewpollock
Labels
cleanupCode hygiene and cleanupdata qualityIssues with data qualityneeds-gcp-infra-accessTo be able to adequately work on this issue will require access to GCP infrastructure

Description

@another-rex
another-rex
opened on Jul 18, 2023

Checks the source field in the database to see if they still exist (does not return 404), if not, mark it as withdrawn.

Activity

  1. another-rex
    added
    good first issueGood for newcomers
    cleanupCode hygiene and cleanup
    and removed
    good first issueGood for newcomers
    on Jul 18, 2023
  2. andrewpollock
    added
    data qualityIssues with data quality
    on Aug 7, 2023

  3. bitwizard25 commented on Aug 18, 2023

    @bitwizard25
    bitwizard25
    on Aug 18, 2023

    Can you assign me this issue i want to work on it

  4. another-rex
    added
    needs-gcp-infra-accessTo be able to adequately work on this issue will require access to GCP infrastructure
    on Dec 5, 2023

  5. another-rex commented on Dec 5, 2023

    @another-rex
    another-rex
    on Dec 5, 2023
    ContributorAuthor

    Hey @rajbhoyar729, sorry missed this comment for a long time. This is more of a internal task as you would need access to the database backend to be able to adequately work on it.

  6. another-rex commented on Dec 5, 2023

    @another-rex
    another-rex
    on Dec 5, 2023
    ContributorAuthor

    #1859 is another example of this being potentially useful.

  7. bitwizard25 commented on Dec 6, 2023

    @bitwizard25
    bitwizard25
    on Dec 6, 2023

    Can you guide me how can I do it

  8. andrewpollock commented on Jan 30, 2024

    @andrewpollock
    andrewpollock
    on Jan 30, 2024
    Contributor

    @rajbhoyar729 your enthusiasm for contributing to OSV has been noticed 🥇
    This particular issue is difficult for external contributors who don't have access to our GCP infrastructure (that's what we cooked up the needs-gcp-infra-access label for).

    Please take a look at the issues tagged with good first issue. These are more suitable for folks like yourself. You may also like to contribute to OSV Scanner

    Stay enthusiastic!

  9. andrewpollock
    self-assigned this
    on Feb 19, 2024
  10. andrewpollock
    added 2 commits that reference this issue on Mar 5, 2024

    Handle GCS bucket record deletion

    9c72b67

    Handle GCS bucket record deletion

    d2ee50c
  11. andrewpollock
    added a commit that references this issue on Apr 24, 2024

    Add a second importer cronjob to run in delete mode

    67a3ade
  12. andrewpollock
    added 2 commits that reference this issue on May 2, 2024

    Handle GCS bucket record deletion (#2030)

    8174d1c

    Add a second importer cronjob to run in delete mode (#2133)

    5aa9653

  13. andrewpollock commented on May 15, 2024

    @andrewpollock
    andrewpollock
    on May 15, 2024
    Contributor

    Manually ran this in Production:

    5220 Bugs in Datastore considered deleted from GCS for cve-osv
23 Bugs in Datastore considered deleted from GCS for debian-dsa
19 Bugs in Datastore considered deleted from GCS for go

  14. andrewpollock commented on May 16, 2024

    @andrewpollock
    andrewpollock
    on May 16, 2024
    Contributor

    I think we can call this done finally.

  15. andrewpollock
    closed this as completedon May 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

cleanupCode hygiene and cleanupdata qualityIssues with data qualityneeds-gcp-infra-accessTo be able to adequately work on this issue will require access to GCP infrastructure

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions