Support `limit` and `last_affected` range events

[nscuro]

I came across the guava vulnerability GHSA-5mg8-w23w-74h3 for which GHSA declares the affected version range as <= 29.0.

In OSV however, this is represented as:

"ranges": [
    {
        "type": "ECOSYSTEM",
        "events": [
            {
                "introduced": "0"
            }
        ]
    }
],
"database_specific": {
    "last_known_affected_version_range": "<= 29.0"
}

Given the constraint <= 29.0, I would've expected the following:

"ranges": [
    {
        "type": "ECOSYSTEM",
        "events": [
            {
                "introduced": "0"
            },
            {
                "last_affected": "29.0"
            }
        ]
    }
]

The current situation makes automated processing unnecessarily hard. If I rely on the ECOSYSTEM range, I'll trigger lots of false positives due to it indicating a >0 constraint. database_specific is not intended to influence vulnerability evaluation according to the spec. This is also visible when inspecting the (auto-generated) Affected versions section on OSV's website: https://osv.dev/vulnerability/GHSA-5mg8-w23w-74h3

At the moment, there are about 1990 advisories affected by this:

$ rg -l '"last_known_affected_version_range"' advisory-database | wc -l
1990

google/osv.dev#474 (comment) already hinted that GHSA currently does not support the limit or last_affected events. Is it planned to be addressed anytime soon?

[chrisbloom7]

@nscuro thank you for bringing this up. The last_affected field is relatively new to the spec and was introduced after our initial build out of our OSV transformers. We have a work item to address this. I've make a note of your ask in our internal issue tracking this work.

[chrisbloom7]

For additional context, the work we have scheduled will allow us to transform upper bound inclusive ranges (i.e. <=) to use the last_affected range event in OSV, however the OSV schema still does not support upper bound exclusive ranges (i.e. <) so there will still be some ranges that continue to use the database specific last_known_affected_version_range when it is determined that using < best represents the upper bound of the advisory based on available info.

[chrisbloom7]

We will also continue to use it for ranges where the upper bound inclusive <= x.y.z does not match the patched version when it's available. This is likely rare, and in this case we will include a fixed range event matching the patched version in the OSV file so you will be able to parse them correctly with automation, but we will still include the last_known_affected_version_range as a way to retain the context of the original advisory for internal purposes. Just want to be clear that there will still be some instances of last_known_affected_version_range remaining even after we switch to using last_affected.

[nscuro]

@chrisbloom7 Thanks so much for the quick and thorough response! Good to know that last_known_affected_version_range will continue to be used in some cases still.

However, I'm wondering whether the ECOSYSTEM range should be omitted if last_known_affected_version_range is used? At least in scenarios like the one described above, where it's both redundant and incomplete next to <= 29.

[Shnatsel]

TL;DR: The non-standard field causes issues for anyone trying to produce or consume OSV data. Please prioritize the migration to the last_affected field from OSV 1.3.

Right now GHSA data exported to OSV causes numerous false positives, if parsed according to the spec. This makes all OSV data about crates.io unusable, unless special measures are taken to filter out GHSA data or support the custom extension field. Which in turn makes any tools consuming crates.io data from OSV unusable at present - which includes big-name tools such as Trivy.

[KateCatlin]

Hi all, thanks for your feedback on this!

We will be prioritizing this update in the coming weeks. I'll post again here when it is shipped!

[KateCatlin]

Hi all, this fix is now shipped!

Existing OSV JSON files won't have the new field yet until they are regenerated, but new ones will start to use it. I'll go ahead and close this issue as a result.

Thanks everyone for driving us forward in resolving this.

[Shnatsel]

Is re-generation of the existing files planned?

It's great that the new files are not affected, but the existing ones still cause false positives when parsed according to the OSV standard.

[KateCatlin]

@Shnatsel yes indeed, we have a nightly job that will resync 25,000 at a time so they'll all be regenerated with the new format over the next week.