UE4: How To Write a Commandlet

Posted on by

Summary

A commandlet in Unreal is a mini-program that can be run from the commandline typically for automation.

A couple of significant examples are:

  • Resave Packages: resaves packages, typically used to resave older packages to bring them up to date.
  • Fixup Redirects: when assets are moved, they leave redirectors behind so that assets that reference them can be updated. Fixup Redirects updates those assets referencing the redirectors, then removes the redirectors that are no longer used.
  • Cooking: convert content into a format more optimized for target platforms.
  • Localization Pipeline: processes text in the game for translation and retrieves translations to include into the game.

How to Create A Commandlet

All commandlets inherit from UCommandlet which provides the basic infrastructure for running a commandlet. As a UObject it automatically gets registered so that it can be found at startup to run.

Typically a commandlet ends in “Commandlet” for discoverability.

Generally commandlets are editor-only and should go into an editor only project.

A commandlet has a Main method that receives commandline parameters and returns an integer exit code that you override. A non-zero exit code is an error and will appear at runtime (particularly relevant in build processes to know something went wrong).

Example:

#pragma once

#include "Commandlets/Commandlet.h"
#include "CommandletSample.generated.h"

UCLASS()
class UCommandletSampleCommandlet : public UCommandlet
{
    GENERATED_BODY()

    virtual int32 Main(const FString& Params) override;
};

The body of a commandlet is like a main function for a standard of C++.

#include "CommandletSample.h"

int32 UCommandletSampleCommandlet::Main(const FString& Params)
{
    return 0;
}

Running a Commandlet

From Visual Studio

You can run a commandlet from Visual Studio by modifying the properties of the project under Debugging→Command Arguments and add a -run= option for your commandlet.

Example:

From Commandline

To run the commandlet from the commandline (such as for Jenkins) you can run:

d:\p4>Engine\Binaries\Win64\UE4Editor-Cmd.exe <Your Project Name> -run=CommandletSample

Parsing Parameters

Commandlets have member functions to assist with parsing parameters by partitioning parameters into tokens and switches:

TArray<FString> Tokens, Switches;
ParseCommandLine(*Params, Tokens, Switches);

To get strings, booleans, and integers, you can use the collection of parsing methods in FParse such as:

bool Value = false;
if(FParse::Bool(*Switches[0], TEXT("boolparam"), &Value)
    && Value)
{
    // Do stuff
}

Output

Output for commandlets is done via UE_LOG. To do so you can declare a log category locally for your commandlet as follows:

DEFINE_LOG_CATEGORY_STATIC(LogCommandletSample, Log, All);

From there you can log as follows:

UE_LOG(LogCommandletSample, Display, TEXT("Found: %s"), *Instance->GetPathName());

Finding & Loading Assets

Commandlets are primarily about automating operations done over some or all assets.

The Asset Registry is the means by which one gets access to the list and operations available for assets in uasset and umap files.

NOTE: A significant limitation of the asset registry is it is limited in accessing assets that are sub-objects in other objects. For example a map can contain actors and those actors may not be returned when querying the asset registry. Instead it may be required to consider assets that can contain your desired asset.

Required headers:

#include "AssetRegistryModule.h"
#include "Developer/AssetTools/Public/AssetToolsModule.h"

Getting access to the Asset Registry:

auto& AssetRegistryModule = FModuleManager::Get().LoadModuleChecked<FAssetRegistryModule>(TEXT("AssetRegistry"));
auto& AssetRegistry = AssetRegistryModule.Get();

To ensure the Asset Registry is up to date use:

// Get all the asset in our content folders
AssetRegistry.SearchAllAssets(true);
while (AssetRegistry.IsLoadingAssets())
{
    AssetRegistry.Tick(1.0f);
}

Then you can query for assets by class (note there are a lot  of other methods in the asset registry to search for assets):

TArray<FAssetData> Assets;
UClass* TargetClass = <Native Class>::StaticClass();
AssetRegistry.GetAssetsByClass(TargetClass->GetFName(), Assets, true);

Once you have your list of assets, you can retrieve them:

for (const FAssetData& Asset : Assets)
{
    <Native Class>* Instance = Cast<Native Class>(Asset.GetAsset());

    // DO STUFF HERE
}

Source Control

You can add, edit, and delete files from source control from commandlets as well.

Source control is available if it has been configured for the user in the project settings or via commandline settings. An example for Perforce is provided:

-SCCProvider=Perforce -P4Port=<your perforce server name and port> -P4User=<p4user> -P4Client=<p4client>

Required headers:

#include "ISourceControlProvider.h"
#include "ISourceControlModule.h"
#include "SourceControlOperations.h"

Verify that source control is available:

SourceControlProvider = &ISourceControlModule::Get().GetProvider();
SourceControlProvider->Init();

if (!SourceControlProvider->IsEnabled())
{
    UE_LOG(LogCommandletSample, Error, TEXT("Source control not initialized. Exiting..."));
    return 2;
}

You can query the source control state of the file using:

FSourceControlStatePtr SourceControlState = SourceControlProvider->GetState(PackageToSave, EStateCacheUsage::ForceUpdate);

It is recommended to force update otherwise cached information may be used that can give erroneous results. There is a version of this method for getting the state of multiple files at once which is far more efficient than one at a time.

Adding, editing, or deleting consists of actions in the source control provider:

const ECommandResult::Type Result = SourceControlProvider->Execute(ISourceControlOperation::Create<FCheckOut>(), PackageToSave);

Saving Packages

Saving a package after modification does require some care.

There are two ways to save packages whether there is a root object (World in maps) or just the package itself.

We use functions like the following to save the package. NOTE: this is a good place to consolidate functionality across uses:

/**
* Helper function to save a package that may or may not be a map package
*
* @param Package The package to save
* @param Filename The location to save the package to
* @param KeepObjectFlags Objects with any these flags will be kept when saving even if unreferenced.
* @param ErrorDevice the output device to use for warning and error messages
* @param LinkerToConformAgainst
* @param optional linker to use as a base when saving Package; if specified, all common names, imports and exports
* in Package will be sorted in the same order as the corresponding entries in the LinkerToConformAgainst

* @return true if successful
*/
bool UGatherDialogCommandlet::SavePackageHelper(UPackage* Package, FString Filename)
{
    Package->FullyLoad();
    EObjectFlags KeepObjectFlags = RF_Standalone;
    FOutputDevice* ErrorDevice = GWarn;
    FLinkerNull* LinkerToConformAgainst = nullptr;
    ESaveFlags SaveFlags = SAVE_None;

    // look for a world object in the package (if there is one, there's a map)
    UWorld* World = UWorld::FindWorldInPackage(Package);
    bool bSavedCorrectly;

    if (World)
    {
      bSavedCorrectly = GEditor->SavePackage(Package, World, RF_NoFlags, *Filename, ErrorDevice, LinkerToConformAgainst, false, true, SaveFlags);
    }
    else
    {
      bSavedCorrectly = GEditor->SavePackage(Package, NULL, KeepObjectFlags, *Filename, ErrorDevice, LinkerToConformAgainst, false, true, SaveFlags);
    }

    // return success
    return bSavedCorrectly;
}

Other Resources

5 thoughts on “UE4: How To Write a Commandlet”

  1. Hi,

    Thanks for the post!

    Could you please help me understanding the RootObject parameter? Somehow I came to a conclusion that this is the object that represents the main asset stored in the package, so it would be a UStaticMesh or a UTexture for a static mesh or a texture respectively.

    I assumed that this parameter is required when saving all kinds of assets, but now your post is saying that it is only required for UWorlds which probably means that my original assumption was wrong.

    I am a bit confused now.

    ~Robert

    Reply ↓

    • Hi Robert,

      As a caveat, I’ll add that a bunch of this is from my experience is from UE3 too, so some usage may have changed (but I don’t think so).

      So there are two interesting flags when it comes to packages (see the EObjectFlags list here: https://docs.unrealengine.com/en-US/API/Runtime/CoreUObject/UObject/EObjectFlags/index.html).

      RF_Public indicates the asset can be referenced outside of its package. If you have two packages A & B with assets x & y respectively (A.x, B.y) and A.x references B.y, if B.y is missing the RF_Public flag, then when you save A, it will complain that it is linked to an external private object.

      The RootObject and RF_Standalone are used to determine what gets saved. Typically one is not set if the other is (though I don’t think there is a check for it).

      When SavePackage goes to determine what assets to save it will go through all top level assets in the package and examine their keep flags. If the asset has all of the keep flags, it will be added to the list of assets to save for the package. This allows a package to have multiple assets at the top level that get saved out that can be referenced (though in UE4 it is more typical to have only one top level asset–UE3 used to have *a lot* more top level assets in a package).

      However if the RootObject is set, it will determine the collection of assets to save based on the graph of assets that RootObject references. So if there is a top level asset in the package with standalone that isn’t referenced, it doesn’t get saved (unless that flag is specified as a keep flag).

      Typically that root object is the UWorld for a map as typically when loading a map, that is the asset it cares about and doesn’t expect to be retrieving anything else.

      Does that help?
      Brent

      Reply ↓

      • Hi Brent,

        Thanks for the reply!

        I did a quick search through the UE4 codebase and it appears to me that what you are saying is correct. I can see that whenever RootObject is not NULL, it is set to an instance of a UWorld.

        I did a further dig into the SavePackage code and discovered that specifying RootObject allows it to call PreSaveRoot and PostSaveRoot on the object which UWorld uses to do whatever it needs to do right before and right after being saved. This seems to be the only reason why UWorlds are saved differently from normal assets.

        What is interesting is that it seems that UE4 packages are still technically able to contain many assets same way UE3 packages did. It seems that the difference in how assets are treated between two engines mainly lies on the level of the editor and the tools.

        Thanks again for being helpful!

        ~Robert

        Reply ↓

  2. A great introduction into commandlets.
    Unfortunately I am completly new to Unreal and commin from a Python Background I don’t understand where I have to put my custom commandlets. I am trying to create a commandlet for creating an unreal project with the help of the commandline…

    Thank you.

    A

    Reply ↓

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.