Skip to content

Support for quantizing activations #2057

@jeyapradhaj

Description

@jeyapradhaj

Hi, I see that the GPTQModel library supports GPTAQ/GPTQ v2. In the GPTAQ paper, results for configurations such as W8A8, W4A4 are shown. However, I could test only weight quantization with GPTAQ/GPTQ v2 so far with the library. Are there any plans to add support for activation quantization as well? Thanks.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions