Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
1 year ago
Solved

Machine Learning Prophet Issues

Good afternoon. I am learning how to use the ML models in Fabric Notebooks but am having issues with Prophet. When I run an expirement using AutoML, it tests multiple models and generally comes back ...
  • v-hashadapu's avatar
    v-hashadapu
    1 year ago

    Hi Anonymous , Thank you for reaching out to the Microsoft Community Forum.

     

    You're getting a KeyError because Prophet in Microsoft Fabric’s AutoML expects a set of engineered features that were automatically created during training but aren’t regenerated at prediction time. Since Fabric doesn’t expose the exact featurization logic, you’ll need to manually recreate those features based on the error message and what the model expects.

     

    First, use MLflow to inspect the required input columns for your model. Example:

    import mlflow

    model = mlflow.pyfunc.load_model("models:/your_model_name/1")

    required_cols = model.metadata.get_input_schema().input_names()

    print("Required columns:", required_cols)

     

    Then, generate the missing features based on your ds datetime column. Example:

    import pandas as pd

    import numpy as np

    df_future = pd.read_csv("your_inference_data.csv")

    df_future['ds'] = pd.to_datetime(df_future['ds'])

    df_future['hour'] = df_future['ds'].dt.hour

    df_future['dayofweek'] = df_future['ds'].dt.dayofweek

    df_future['dayofyear'] = df_future['ds'].dt.dayofyear

    df_future['month'] = df_future['ds'].dt.month

    df_future['quarter'] = df_future['ds'].dt.quarter

    df_future['minute'] = df_future['ds'].dt.minute

    df_future['second'] = df_future['ds'].dt.second

    def add_cyclic_features(df, col, max_val, prefix):

        df[f'{prefix}_sin'] = np.sin(2 * np.pi * df[col] / max_val)

        df[f'{prefix}_cos'] = np.cos(2 * np.pi * df[col] / max_val)

        return df

    df_future = add_cyclic_features(df_future, 'hour', 24, 'ds_hour')

    df_future = add_cyclic_features(df_future, 'dayofweek', 7, 'ds_dayofweek')

    df_future = add_cyclic_features(df_future, 'dayofyear', 365, 'ds_dayofyear')

    df_future = add_cyclic_features(df_future, 'month', 12, 'ds_month')

    df_future = add_cyclic_features(df_future, 'quarter', 4, 'ds_quarter')

    df_future = add_cyclic_features(df_future, 'minute', 60, 'ds_minute')

    df_future = add_cyclic_features(df_future, 'second', 60, 'ds_second')

    for i in range(1, 5):

        df_future[f'ds_sin{i}'] = np.sin(i * 2 * np.pi * df_future['ds'].dt.dayofyear / 365)

        df_future[f'ds_cos{i}'] = np.cos(i * 2 * np.pi * df_future['ds'].dt.dayofyear / 365)

     

    Validate that all required columns are present before predicting. Example:

    missing_cols = [col for col in required_cols if col not in df_future.columns]

    if missing_cols:

        raise ValueError(f"Missing columns: {missing_cols}")

     

    Then make predictions. Example:

    forecast = model.predict(df_future[required_cols])

    print(forecast[['ds', 'yhat', 'yhat_lower', 'yhat_upper']])

     

    If this helped solve the issue, please consider marking it “Accept as Solution” and giving a ‘Kudos’ so others with similar queries may find it more easily. If not, please share the details, always happy to help.
    Thank you.