Feat: Add support for additional metapackages (closes #238)

[rhoerr]

Resolves #238

Overview

Substantial refactor replacing hardcoded metapackage flags with flexible configuration system.

Key Changes

Configuration Architecture

• Removed: Boolean flags magentoCommunityEditionProject and magentoCommunityEditionMetapackage from repositoryBuildDefinition
• Added: extraMetapackages array accepting metapackage definitions with:
  • name, type (project/metapackage), description
  • fromTag (optional starting version)
  • transform[] (array of transformation functions)

Refactored Methods

• Eliminated specialized functions:

  • createMagentoCommunityEditionMetapackage()
  • createMagentoCommunityEditionProject()
  • createProjectPackagesSinceTag()
  • createMagentoCommunityEditionMetapackagesSinceTag()

• Replaced with generic createMetaPackagesSinceTag() that iterates extraMetapackages[]

Transform Functions

New dedicated modules for composition-based transforms:

• src/build-metapackage/magento-community-edition.js
  • transformMagentoCommunityEditionProject()
  • transformMagentoCommunityEditionProduct()
• src/build-metapackage/mage-os-community-edition.js
  • transformMageOSCommunityEditionProject()
  • transformMageOSCommunityEditionProduct()

Transforms now async, called sequentially: for (const fn of metapackage.transform) { composerConfig = await fn(...); }

Configuration Examples

packages-config.js (Magento):

extraMetapackages: [
  {
    name: 'project-community-edition',
    type: 'project',
    transform: [transformMagentoCommunityEditionProject]
  },
  {
    name: 'product-community-edition',
    type: 'metapackage',
    transform: [transformMagentoCommunityEditionProduct]
  }
]

mageos-release-build-config.js (Mage-OS - chained transforms):

transform: [
  transformMagentoCommunityEditionProject,
  transformMageOSCommunityEditionProject
]

Context Propagation

• releaseContext (buildState with composerRepoUrl) now passed to processMirrorInstruction() and downstream
• Removed global setMageosPackageRepoUrl() in favor of explicit context

Historical Data

Added magento_version to Mage-OS product metapackage history files for upstream reference tracking.

Benefits

• Extensible: Add metapackages via config without code changes
• Composable: Stack multiple transforms for metapackage needs
• Consistent: Same createMetaPackagesSinceTag() handles all metapackages
• Maintainable: Transform logic isolated in dedicated modules

Additional notes

Build issues

In the process of refactoring and testing, I found some issues with the prior build setup that this fixes.

• project packages weren't being given the proper separate description to match upstream
• Mirroring was setting minimum-stability to stable even for alpha/beta releases
• Mirroring previously set magento/magento2-base requirement version to null; it now properly matches the release version
• Mage-OS new versions have included magento_version in the product composer.json for reference info, but historical Mage-OS versions have not, meaning each new release removed the info from the prior one. That'll no longer be the case. I added it to the release history file, and set it to copy extra data in too if present.

<   "version": "1.3.1"
---
>   "version": "1.3.1",
>   "extra": {
>     "magento_version": "2.4.8-p2"
>   }

Adding metapackages

The ultimate point of this was to be able to add more metapackages to the Mage-OS release process, as situations may warrant.

At least for now, I opted to do it via transform functions, rather than some system of include or exclude or add lists.

This is limited to changes to project and product composer files. If we want different code for one versus another, it would require a separate fork or bigger changes where packages are actually built.

I tested with these added metapackages, purely for demonstration:

      {
        name: 'project-minimal',
        type: 'project',
        fromTag: '3.0.0',
        description: 'Mage-OS Minimal Edition Project',
        transform: [
          transformMagentoCommunityEditionProject,
          transformMageOSCommunityEditionProject,
        ]
      },
      {
        name: 'product-minimal',
        type: 'metapackage',
        fromTag: '3.0.0',
        description: 'Mage-OS Minimal Edition',
        transform: [
          transformMagentoCommunityEditionProduct,
          transformMageOSCommunityEditionProduct,
          /**
           * @param {{}} composerConfig 
           * @param {repositoryBuildDefinition} instruction 
           * @param {metapackageDefinition} metapackage
           * @param {buildState} release 
           */
          async (composerConfig, instruction, metapackage, release) => {
            // Mock removing inventory and graphql packages for minimal edition
            for (const pkg of Object.keys(composerConfig.require)) {
              if (pkg.includes('-graph-ql') || pkg.includes('-inventory')) {
                delete composerConfig.require[pkg];
              }
            }
            
            return composerConfig;
          }
        ]
      }

[rhoerr]

Remembered an issue I need to address before finalizing: I have not yet tested historical builds of alternate metapackage releases. I suspect changes will be needed for the transform changes to be locked in place for historical releases, and not reapplied from the transform against past releases. We don't want v3 removing additional packages to also remove those from v2.

[rhoerr]

Handled. New 'minimal' example transforms:

      {
        name: 'project-minimal',
        type: 'project',
        fromTag: '2.1.0',
        description: 'Mage-OS Minimal Edition Project',
        transform: [
          transformMagentoCommunityEditionProject,
          transformMageOSCommunityEditionProject,
        ]
      },
      {
        name: 'product-minimal',
        type: 'metapackage',
        fromTag: '2.1.0',
        description: 'Mage-OS Minimal Edition',
        transform: [
          transformMagentoCommunityEditionProduct,
          transformMageOSCommunityEditionProduct,
          /**
           * @param {{}} composerConfig 
           * @param {repositoryBuildDefinition} instruction 
           * @param {metapackageDefinition} metapackage
           * @param {buildState} release 
           */
          async (composerConfig, instruction, metapackage, release) => {
            if (composerConfig.version !== release.version) {
              return composerConfig;
            }

            // Mock removing inventory and graphql packages for minimal edition
            for (const pkg of Object.keys(composerConfig.require)) {
              if (pkg.includes('-graph-ql') || pkg.includes('-inventory') || pkg.includes('-media')) {
                delete composerConfig.require[pkg];
              }
            }
            
            return composerConfig;
          }
        ]
      }

Added code to the metapackage builders to allow us to store the entire built JSON in the repo, instead of just the composer template values for the version, to account for any transform differences in effect as well.

It is necessary to create new composer-templates folders and files for each separate metapackage. This makes sense to do anyway.

It is necessary to commit the entire composer project and product JSON files for the history for these deviating metapackages. We have to track the full state to keep a reliable history.

Long-term, we could perhaps simplify by making release creation and repo/mirror building two separate things, where the new-version build tags and PRs the various files and updates involved, and the repo/mirror build purely creates files based on version histories and tags. But it's probably not worth the effort any time soon.

As it stands, I think this is ready.