Tutorial 4: Using the RTA Micro-service#

This tutorial builds upon Tutorial 3. Here, the filtered and decimated rates and pressure values will be uploaded to a Topaze document (already uploaded to the well)and a new model will be executed using the RTA micro-service.

This tutorial emphasizes the use of the micro-service, the extraction of the simulation resultsand the updating of model parameters.

The code for Tutorial 4 can be found in this zip file.

An additional input - Topaze file#

There is one additional input for this tutorial along with what is shown in Tutorial 3.

Listing 28 The Topaze file input is represented by the following YAML code#

    Topaze file:
      type: file
      readOnly: false
      mandatory: true

User task script: Specifying the model#

The Topaze file is represented by a kappa_sdk.Document instance. The model input is described in xml format. The kappa_sdk.Document.set_model_xml() method accepts the xml file representing the model input (or optionally, only the input which is different in the Topaze model in the kappa_sdk.Document instance)and then runs a “Generate” or an “Improve”.

An example is shown in the code snippet in Listing 30. A kappa_sdk.ModelParser instance parses an xml and sets new parameter values using kappa_sdk.ModelParser.set_parameter_value(). The method kappa_sdk.ModelParser.export() creates the new xml, which is sent to the kappa_sdk.Document instance (“topaze_doc”) using kappa_sdk.Document.set_model_xml(). This method will run a “Generate” (by default) and give a return message if the execution is not successful.

Listing 30 Section of code showing the modification of a model xml and re-injection of that xml into a Topaze document.#

...
delta, k, xmf = x

parser = ModelParser(model_xml)
parser.set_parameter_value(topaze_doc.analyses[0].id, "KWKA_RES_PAR",
                           {"Type": "FRACTIONAL_DIMENSION", "ZoneIndexX": "1"}, str(delta))
k = np.power(10, k)
internal_perm = services.unit_converter.convert_to_internal(UnitEnum.perm_milli_darcy, k)
parser.set_parameter_value(topaze_doc.analyses[0].id, "KWKA_RES_PAR",
                           {"Type": "PERMEABILITY"}, str(internal_perm))
xmf = np.power(10, xmf)
internal_xmf = services.unit_converter.convert_to_internal(UnitEnum.length_feet, xmf)
parser.set_parameter_value(topaze_doc.analyses[0].id, "KWKA_WELL_PAR",
                           {"Type": "FRACTURE_XF"}, str(internal_xmf))

# get the updated xml
new_model_xml = parser.export()
ret = topaze_doc.set_model_xml(new_model_xml)

if not ret.is_success:
    services.log.error(ret.message)
...

The file “model_xml” can be retrieved preceding Listing 30 using kappa_sdk.Document.get_model_xml().

# Retrieve the model xml file
model_xml = topaze_doc.get_model_xml()

Now that we have our code to modify a model (via the xml), re-run the modeland extract the results, we can do something interesting such as run a least squares optimization using SciPy. After the import statements we have three functions: the first updates the model xml and runs the RTA micro-service, the second calculates the least squares residualand the third runs the least squares optimization. Following these function definitions, the Python code from Tutorial 2 is repeated (without the plotting). Finally, we have the code where the model xml is extracted and the optimization code is called.

Plotting the regression results#

We are going to plot the results of the least squares regression by extracting them from the model and not from the output of a user task. In this case, we must refresh the plot programmatically within the user task. The first step is to check if the plot has already been created. If so, then the plot can be retrieved, deleted and re-created. See Listing 31

Listing 31 Updating a plot#

this_task = services.user_task
plot_name = 'Regression Summary'
plot = next((x for x in this_task.plots if x.name == plot_name), None)
if plot is not None:
    plot.delete()

Then, we pass the data to the plot in the form of a list containing the x,y points.

Listing 32 Creating the data list#

#  Setting up data for plotting
data_measure = 'LiquidRateSurface'
historical_name = 'Decimated rates'
simulated_name = 'Simulated oil rate'
(data_times, data_values), (sim_times_init, sim_init) = run_topaze(x_init, model_xml, topaze_doc, data_measure, historical_name, simulated_name)
date_list = list()
for x in data_times:
    new_date = first_date + datetime.timedelta(hours=x)
    date_list.append(new_date)
sim_init_list = list()
for x in sim_times_init:
    new_date = first_date + datetime.timedelta(hours=x)
    sim_init_list.append(new_date)

my_data_type = PlotDataTypesEnum.oil_rate_surface
data_series = summary_plot.add_embedded_data(Vector(date_list, data_values, first_date), 'Data', pane_name, show_symbols=False,
                                             first_x=first_date, show_lines=True, data_type=my_data_type, is_by_step=True, use_elapsed=False)
data_series.set_lines_aspect(style=LineAspectEnum.dash)
init_series = summary_plot.add_embedded_data(Vector(sim_init_list, sim_init, first_date), 'Initial', pane_name, show_symbols=False,
                                             first_x=first_date, show_lines=True, data_type=my_data_type, is_by_step=True, use_elapsed=False)
init_series.set_lines_aspect(style=LineAspectEnum.dot)

The plot is then updated in the usual fashion by kappa_sdk.Plot.update_plot().

The full Python script#

The full user task script is shown below in Listing 33.

Note

Running a least squares optimization in this manner will be much slower than running an “Improve” within the RTA micro-service. This example is given for demonstration purposes only.

Listing 33 tutorial_4.py#

from kappa_sdk.user_tasks.user_task_environment import services
import tutorial_4_utility as utility
import datetime
from kappa_sdk import KWModuleEnum, UnitEnum
from kappa_sdk import ModelParser, Document
import numpy as np
import numpy.typing as npt
from scipy.interpolate import interp1d
from scipy import optimize
import math
from kappa_sdk import Plot, PlotDataTypesEnum, LineAspectEnum, Vector
from typing import Tuple, List, cast
from uuid import UUID


def run_topaze(x: Tuple[float, float, float], model_xml: str, topaze_doc: Document, data_measure: str, historical_name: str, simulated_name: str) \
        -> Tuple[Tuple[List[float], List[float]], Tuple[List[float], List[float]]]:
    delta, k, xmf = x

    parser = ModelParser(model_xml)
    parser.set_parameter_value(topaze_doc.analyses[0].id, "KWKA_RES_PAR",
                               {"Type": "FRACTIONAL_DIMENSION", "ZoneIndexX": "1"}, str(delta))
    k = np.power(10, k)
    internal_perm = services.unit_converter.convert_to_internal(UnitEnum.perm_milli_darcy, k)
    parser.set_parameter_value(topaze_doc.analyses[0].id, "KWKA_RES_PAR",
                               {"Type": "PERMEABILITY"}, str(internal_perm))
    xmf = np.power(10, xmf)
    internal_xmf = services.unit_converter.convert_to_internal(UnitEnum.length_feet, xmf)
    parser.set_parameter_value(topaze_doc.analyses[0].id, "KWKA_WELL_PAR",
                               {"Type": "FRACTURE_XF"}, str(internal_xmf))

    # get the updated xml
    new_model_xml = parser.export()
    result = topaze_doc.set_model_xml(new_model_xml)

    if not result.is_success:
        services.log.error(result.message)
        raise ValueError(f"Document model update has failed: {result.message}")

    # retrieve generated data from the document
    vectors = topaze_doc.analyses[0].get_plot_data('History')  # <-- need to document plot types
    #  NOTE:  dates coming from vectors is not correct because the reference time is often not initialized correctly
    for v in vectors:
        if v.data_measure == data_measure and v.data_name == historical_name:
            data = v.values
            data_times = v.elapsed_times
        elif v.data_measure == data_measure and v.data_name == simulated_name:
            sim = v.values
            sim_times = v.elapsed_times

    if data is None or sim is None:
        raise Exception('No data or simulated data found in the document')

    return (data_times, data), (sim_times, sim)


def get_ls_residual(x: Tuple[float, float, float], model_xml: str, topaze_doc: Document, data_measure: str, historical_name: str, simulated_name: str) -> npt.NDArray[np.float64]:

    (data_times, data), (sim_times, sim) = run_topaze(x, model_xml, topaze_doc, data_measure, historical_name, simulated_name)
    sim_interpolator = interp1d(sim_times, sim, bounds_error=False, fill_value='extrapolate')
    data_times_array = np.array(data_times)
    data_array = np.array(data)
    start_match_time = 1000  # started simulation at 1000 hours
    diff: npt.NDArray[np.float64] = sim_interpolator(data_times_array[data_times_array > start_match_time]) - data_array[data_times_array > start_match_time]
    return diff


def perform_least_squares(model_xml: str, topaze_doc: Document, x_init: Tuple[float, float, float], min_vals: npt.NDArray[np.float64], max_vals: npt.NDArray[np.float64], data_measure: str, historical_name: str, simulated_name: str) -> optimize.OptimizeResult:

    args_least_squares = [model_xml, topaze_doc, data_measure, historical_name, simulated_name]
    result = optimize.least_squares(get_ls_residual, x_init, bounds=(min_vals, max_vals),
                                    args=args_least_squares, method='dogbox')
    return result


services.log.info('[Tutorial 3] Start of user task')
data = services.data_enumerator.to_enumerable([cast(UUID, services.parameters.input['Rate']), cast(UUID, services.parameters.input['Pressure'])])
services.log.info('[Tutorial 3] Data read by data_enumerator')

dates = list()
rates = list()
pressures = list()
times = list()
first_date = data[0].date
for point in data:
    dates.append(point.date)
    elapsed = (point.date - first_date).total_seconds() / 60.0 / 60.0
    times.append(elapsed)
    rates.append(point.values[0])
    pressures.append(point.values[1])

#  Size of window for smoothing
window_size = cast(int, services.parameters.input['Window size'])
services.log.info('[Tutorial 4 Smoothing pressure signal for non-zero rate periods')
filtered_pressures = utility.filter_positive_values(rates, pressures, window_size)

#  Time period for rate decimation
delta_t_max = cast(float, services.parameters.input['Time decimation'])

# # Decimate the rate values by delta_t_max (preserving cumulative production)
services.log.info('[Tutorial 4] Decimate the rate values by {}'.format(delta_t_max))
decimated_times, decimated_rates = utility.get_decimated_times_rates(times, rates, delta_t_max)

decimated_dates = list()
for t in decimated_times:
    new_date = first_date + datetime.timedelta(hours=t)
    decimated_dates.append(new_date)

services.log.info('[Tutorial 4] Interpolate pressure')
pressure_interpolator = interp1d(times, filtered_pressures)
filtered_pressures = (pressure_interpolator(decimated_times))
services.data_command_service.replace_data(str(services.parameters.output['Filtered pressure']), decimated_dates, filtered_pressures.tolist())

#  Step data requires the set_first_x method
services.data_command_service.set_first_x(str(services.parameters.output['Decimated rates']), first_date)
services.data_command_service.replace_data(str(services.parameters.output['Decimated rates']), decimated_dates[1:], decimated_rates[1:])

services.log.info('[Tutorial 4] Getting Topaze document')
file_id = services.parameters.input['Topaze file']
topaze_doc = next(x for x in services.well.documents if (x.type == KWModuleEnum.topaze and x.file_id == str(file_id)))

# Set new pressure and rate values
ret = topaze_doc.reset_pressure(str(services.parameters.output['Filtered pressure']))
if not ret.is_success:
    services.log.error(ret.message)
ret = topaze_doc.reset_rate(str(services.parameters.output['Decimated rates']), 'OilRate')
if not ret.is_success:
    services.log.error(ret.message)

services.log.info('[Tutorial 4] Retrieving model xml')
# get the model XML once and re-use it on each iteration
model_xml = topaze_doc.get_model_xml()

services.log.info('[Tutorial 4] Reset delta, k, xmf')
#  Initial values and min/max bounds
delta = 0.45
k = math.log10(1E-4)  # Working in log(k) space
xmf = math.log10(5000)  # Working in log(Xmf) space
x_init = (delta, k, xmf)
min_vals = np.array([0.2, -5, 3])
max_vals = np.array([0.7, -3, 5])

this_task = services.user_task
plot_name = 'Regression Summary'
plot = next((x for x in this_task.plots if x.name == plot_name), None)
if plot is not None:
    plot.delete()

pane_name = 'Init vs Final Match'
summary_plot: Plot = services.user_task.create_plot(plot_name, pane_name)

#  Setting up data for plotting
data_measure = 'LiquidRateSurface'
historical_name = 'Decimated rates'
simulated_name = 'Simulated oil rate'
(data_times, data_values), (sim_times_init, sim_init) = run_topaze(x_init, model_xml, topaze_doc, data_measure, historical_name, simulated_name)
date_list = list()
for x in data_times:
    new_date = first_date + datetime.timedelta(hours=x)
    date_list.append(new_date)
sim_init_list = list()
for x in sim_times_init:
    new_date = first_date + datetime.timedelta(hours=x)
    sim_init_list.append(new_date)

my_data_type = PlotDataTypesEnum.oil_rate_surface
data_series = summary_plot.add_embedded_data(Vector(date_list, data_values, first_date), 'Data', pane_name, show_symbols=False,
                                             first_x=first_date, show_lines=True, data_type=my_data_type, is_by_step=True, use_elapsed=False)
data_series.set_lines_aspect(style=LineAspectEnum.dash)
init_series = summary_plot.add_embedded_data(Vector(sim_init_list, sim_init, first_date), 'Initial', pane_name, show_symbols=False,
                                             first_x=first_date, show_lines=True, data_type=my_data_type, is_by_step=True, use_elapsed=False)
init_series.set_lines_aspect(style=LineAspectEnum.dot)

# # Least squares optimization
services.log.info('[Tutorial 4] Performing least squares')
result = perform_least_squares(model_xml, topaze_doc, x_init, min_vals, max_vals, data_measure, historical_name, simulated_name)
services.log.info('[Tutorial 4] Converged to {}'.format(result.x))
services.log.info('[Tutorial 4] Rerunning model at optimal point')

# Run final result and add that to plot
(data_times, data_values), (sim_times_final, sim_final) = run_topaze(result.x, model_xml, topaze_doc, data_measure, historical_name, simulated_name)
sim_final_list = list()
for x in sim_times_final:
    new_date = first_date + datetime.timedelta(hours=x)
    sim_final_list.append(new_date)

new_final_vector = Vector(sim_final_list, sim_final, first_date)
final_series = summary_plot.add_embedded_data(new_final_vector, 'Final', pane_name, show_symbols=False, show_lines=True,
                                              first_x=first_date, data_type=my_data_type, is_by_step=True, use_elapsed=False)
final_series.set_lines_aspect('Purple', 4, LineAspectEnum.solid)

summary_plot.update_plot()

We can see the results of the user task in the before (Fig. 26) and after (Fig. 27) images below.

../_images/tutorial_4_before.png — Fig. 26 The original mismatch of the oil (“Kite - qo”, green dashed line) and simulation results (“Simulated oil rate”, solid green line) before the execution of the user task.#

../_images/tutorial_4_after.png — Fig. 27 The final mismatch of the oil (“Kite - qo”, green dashed line) and simulation results (“Simulated oil rate”, solid green line) after the execution of the user task.#

The simulation file is given below. It is similar to what is found in Tutorial 2, with additional code to initialize the input for the Topaze document.