Writing Instances: Upsert¶

We assume that you have generated a SDK for the WindTurbine model and have a client ready to go.

The SDK generated by SDK supports nested upsert

In [3]:

from wind_turbine import WindTurbineClient

from wind_turbine import WindTurbineClient

In [4]:

pygen = WindTurbineClient.from_toml("config.toml")

pygen = WindTurbineClient.from_toml("config.toml")

Constructing new Turbine¶

When constructing a new WindTurbine we need to use the generated data classes from pygen. We can import this as follows:

In [7]:

from wind_turbine import data_classes as data_cls

from wind_turbine import data_classes as data_cls

The location of the data classes is determined by the parameter top_level_package which is set when you generate the SDK, either using the generate_sdk_notebook (simplified wrapper around) or generate_sdk. If you don't set it, it will be default set to the external_id of the data model converted to snake_case. For this example, the external_id=WindTurbine thus the top_level_package = wind_turbine.

Lets construct a new windmill with TimeSeries. Note the example below is not complete (some TimeSeries and components are missing), but is kept short to make it easier to grasp

In [8]:

new_turbine = data_cls.WindTurbineWrite(
    external_id="windmill:demo",
    capacity=10.0,
    windfarm="Fornebu",
    name="Windmill ATH",
    rotor=data_cls.RotorWrite(
        external_id="windmill:demo:rotor",
    ),
    nacelle=data_cls.NacelleWrite(
        external_id="windmill:demo:nacelle",
    ),
    blades=[
        data_cls.BladeWrite(
            external_id="windmill:demo:blade1",
            is_damaged=False,
            name="Blade 1",
        ),
        data_cls.BladeWrite(
            external_id="windmill:demo:blade2",
            is_damaged=False,
            name="Blade 2",
        ),
        data_cls.BladeWrite(
            external_id="windmill:demo:blade3",
            is_damaged=True,
            name="Blade 3",
        ),
    ],
)

new_turbine = data_cls.WindTurbineWrite( external_id="windmill:demo", capacity=10.0, windfarm="Fornebu", name="Windmill ATH", rotor=data_cls.RotorWrite( external_id="windmill:demo:rotor", ), nacelle=data_cls.NacelleWrite( external_id="windmill:demo:nacelle", ), blades=[ data_cls.BladeWrite( external_id="windmill:demo:blade1", is_damaged=False, name="Blade 1", ), data_cls.BladeWrite( external_id="windmill:demo:blade2", is_damaged=False, name="Blade 2", ), data_cls.BladeWrite( external_id="windmill:demo:blade3", is_damaged=True, name="Blade 3", ), ], )

When writing nested data we can specify edges either with an external id for the end node, or another data class.

The advangage of using a nested data class is that we can express edges without being explicit. In the example above, we are able to express that the blades Blade 1-3 are connected to the windmill windmill ATH and that the windmill ATH is also linked to a nacelle and rotor.

Inspecting Resources to create¶

We can inspect the nodes, edges and other resources that will be created by using the .to_instances_write on the new wind turbine object.

In [11]:

resources = new_turbine.to_instances_write()

resources = new_turbine.to_instances_write()

In [12]:

len(resources.nodes), len(resources.edges), len(resources.time_series)

len(resources.nodes), len(resources.edges), len(resources.time_series)

Out[12]:

(4, 0, 0)

In [13]:

resources.nodes

resources.nodes

Out[13]:

	space	external_id	instance_type	sources
0	sp_wind	windmill:demo	node	[{'properties': {'blades': [{'space': 'sp_wind...
1	sp_wind	windmill:demo:blade1	node	[{'properties': {'is_damaged': False, 'name': ...
2	sp_wind	windmill:demo:blade2	node	[{'properties': {'is_damaged': False, 'name': ...
3	sp_wind	windmill:demo:blade3	node	[{'properties': {'is_damaged': True, 'name': '...

In [14]:

resources.edges

resources.edges

Out[14]:

In [15]:

resources.time_series

resources.time_series

Out[15]:

Creating new WindTurbine¶

Optinal Reading: Why pygen.upsert and not pygen.wind_turbine.upsert?

In contrast from other methods, the .upsert method is on the pygen instead of the individual API class. So instead of pygen.wind_turbine.upsert, we use pygen.upsert.

The reason for this is that the new_turbine we created above is enhanced by pygen with all the information needed to write it correctly to our data model. This means that all .upsert methods are the same, this is in contrast to methods such as .list and .retrieve which are specialized for each data type.

Furthermore, the reason for not duplicating the .upsert methods on each API class (pygen.wind_turbine.upsert, pygen.blade.upsert, and so on) is that encourages an anti-pattern (bad practice), in which nodes and edges are created in small batches. It is much more efficient to create all nodes and edges in as few batches as possible.

In [16]:

created = pygen.upsert(new_turbine)

created = pygen.upsert(new_turbine)

Note that the call above created 4 nodes. If you had linked metmasts, it would have created edges as well.

In [17]:

created.nodes

created.nodes

Out[17]:

	space	instance_type	external_id	version	was_modified	last_updated_time	created_time
0	sp_wind	node	windmill:demo	1	True	2024-11-16 17:30:21.812	2024-11-16 17:30:21.812
1	sp_wind	node	windmill:demo:blade1	1	True	2024-11-16 17:30:21.812	2024-11-16 17:30:21.812
2	sp_wind	node	windmill:demo:blade2	1	True	2024-11-16 17:30:21.812	2024-11-16 17:30:21.812
3	sp_wind	node	windmill:demo:blade3	1	True	2024-11-16 17:30:21.812	2024-11-16 17:30:21.812

In [18]:

created.edges

created.edges

Out[18]:

In [19]:

created.time_series

created.time_series

Out[19]:

We can inspect the newly created windmill by calling retrieve with the external id

In [20]:

pygen.wind_turbine.retrieve(new_turbine.external_id)

pygen.wind_turbine.retrieve(new_turbine.external_id)

Out[20]:

	value
space	sp_wind
external_id	windmill:demo
data_record	{'version': 1, 'last_updated_time': 2024-11-16...
node_type	None
capacity	10.0
description	None
name	Windmill ATH
blades	[windmill:demo:blade1, windmill:demo:blade2, w...
datasheets	[]
metmast	None
nacelle	windmill:demo:nacelle
power_curve	None
rotor	windmill:demo:rotor
windfarm	Fornebu

In [ ]:

Upsert Parameters replace, write_none, and allow_version_increase¶

The upsert method have several parameters that control how the upsert call should be done. In this section, we will go through each of these flags.

Parameter: replace¶

The replace flag decide what to do if the item we are upserting already exists. If replace is set to True all properties of the existing item will be replaced by the properties set in the upsert call, and the properties not included will be to null. If replace is set to False, then only the properties included in the upsert call will be updated.

Let's demonstrate this by creating a new blade and update it. We start by creating a new blade and call upsert on it.

A blade has two properties: name and is_damaged. In addition, it has edges to the sensor_positions connected to the blade. In this example, we will focus on the two properties, which are both nullable, meaning that they are optional.

In [22]:

from wind_turbine import data_classes as data_cls

from wind_turbine import data_classes as data_cls

In [23]:

new_blade = data_cls.BladeWrite(
    external_id="windmill:demo:blade4",
    name="Demo Blade",
)

new_blade = data_cls.BladeWrite( external_id="windmill:demo:blade4", name="Demo Blade", )

In [24]:

created_blade = pygen.upsert(new_blade)

created_blade = pygen.upsert(new_blade)

In [25]:

created_blade.nodes

created_blade.nodes

Out[25]:

	space	instance_type	external_id	version	was_modified	last_updated_time	created_time
0	sp_wind	node	windmill:demo:blade4	1	True	2024-11-16 17:32:28.380	2024-11-16 17:32:28.380

In [26]:

retrieved_blade = pygen.blade.retrieve(new_blade.external_id)

retrieved_blade = pygen.blade.retrieve(new_blade.external_id)

In [27]:

retrieved_blade

retrieved_blade

Out[27]:

	value
space	sp_wind
external_id	windmill:demo:blade4
data_record	{'version': 1, 'last_updated_time': 2024-11-16...
node_type	None
is_damaged	None
name	Demo Blade
sensor_positions	None

We can see that the is_damaged property is not set, while we have the name property set. We will now update the blade with the is_damaged property set to True.

In [28]:

updated_blade = data_cls.BladeWrite(
    external_id="windmill:demo:blade4",
    is_damaged=True,
)

updated_blade = data_cls.BladeWrite( external_id="windmill:demo:blade4", is_damaged=True, )

In [29]:

update = pygen.upsert(updated_blade, replace=False)

update = pygen.upsert(updated_blade, replace=False)

In [30]:

update.nodes

update.nodes

Out[30]:

	space	instance_type	external_id	version	was_modified	last_updated_time	created_time
0	sp_wind	node	windmill:demo:blade4	2	True	2024-11-16 17:32:42.954	2024-11-16 17:32:28.380

In [31]:

retrieved_blade = pygen.blade.retrieve(new_blade.external_id)

retrieved_blade = pygen.blade.retrieve(new_blade.external_id)

In [32]:

retrieved_blade

retrieved_blade

Out[32]:

	value
space	sp_wind
external_id	windmill:demo:blade4
data_record	{'version': 2, 'last_updated_time': 2024-11-16...
node_type	None
is_damaged	True
name	Demo Blade
sensor_positions	None

We see that the blade property is_damaged is now set to True, while the name property is unchanged. This is because we set replace to False. If we set replace to True, then the name property would be set to null. Let's create a new update were we set the is_damaged property to False and use replace set to True.

In [33]:

blade_update2 = data_cls.BladeWrite(
    external_id="windmill:demo:blade4",
    is_damaged=False,
)

blade_update2 = data_cls.BladeWrite( external_id="windmill:demo:blade4", is_damaged=False, )

In [34]:

update2 = pygen.upsert(blade_update2, replace=True)

update2 = pygen.upsert(blade_update2, replace=True)

In [35]:

update2.nodes

update2.nodes

Out[35]:

	space	instance_type	external_id	version	was_modified	last_updated_time	created_time
0	sp_wind	node	windmill:demo:blade4	3	True	2024-11-16 17:33:09.136	2024-11-16 17:32:28.380

In [36]:

retrieved_blade = pygen.blade.retrieve(new_blade.external_id)

retrieved_blade = pygen.blade.retrieve(new_blade.external_id)

In [37]:

retrieved_blade

retrieved_blade

Out[37]:

	value
space	sp_wind
external_id	windmill:demo:blade4
data_record	{'version': 3, 'last_updated_time': 2024-11-16...
node_type	None
is_damaged	False
name	None
sensor_positions	None

Notice that the name property is now set to null as we set replace to True.

In [38]:

# Cleanup
pygen.delete(new_blade.external_id)

# Cleanup pygen.delete(new_blade.external_id)

Out[38]:

InstancesDeleteResult(nodes=[NodeId(space='sp_wind', external_id='windmill:demo:blade4')], edges=[])

Parameter: write_none¶

By default, when calling .upsert properties which are set to None are not sent to the server. This is because we interpret None as "skip this property". If you want to set a property to null you can use the write_none flag.

Let's demonstrate this by creating a new blade with the is_damaged property set to True, and then update it to be not set.

In [46]:

new_blade = data_cls.BladeWrite(external_id="windmill:demo:write_none", name="Demo Blade", is_damaged=True)

new_blade = data_cls.BladeWrite(external_id="windmill:demo:write_none", name="Demo Blade", is_damaged=True)

In [47]:

pygen.upsert(new_blade).nodes

pygen.upsert(new_blade).nodes

Out[47]:

	space	instance_type	external_id	version	was_modified	last_updated_time	created_time
0	sp_wind	node	windmill:demo:write_none	1	True	2024-11-16 17:34:18.147	2024-11-16 17:34:18.147

In [48]:

retrieved_blade = pygen.blade.retrieve(new_blade.external_id)
retrieved_blade

retrieved_blade = pygen.blade.retrieve(new_blade.external_id) retrieved_blade

Out[48]:

	value
space	sp_wind
external_id	windmill:demo:write_none
data_record	{'version': 1, 'last_updated_time': 2024-11-16...
node_type	None
is_damaged	True
name	Demo Blade
sensor_positions	None

In [49]:

updated_blade = data_cls.BladeWrite(
    external_id="windmill:demo:write_none",
    is_damaged=None,
)

updated_blade = data_cls.BladeWrite( external_id="windmill:demo:write_none", is_damaged=None, )

In [50]:

pygen.upsert(updated_blade, write_none=False).nodes

pygen.upsert(updated_blade, write_none=False).nodes

Out[50]:

In [51]:

retrieved_blade = pygen.blade.retrieve(new_blade.external_id)
retrieved_blade

retrieved_blade = pygen.blade.retrieve(new_blade.external_id) retrieved_blade

Out[51]:

	value
space	sp_wind
external_id	windmill:demo:write_none
data_record	{'version': 1, 'last_updated_time': 2024-11-16...
node_type	None
is_damaged	True
name	Demo Blade
sensor_positions	None

We see that when we set the is_damaged property to None it is not sent to the server. If we set write_none to True then the is_damaged property would be set to null.

In [52]:

pygen.upsert(updated_blade, write_none=True).nodes

pygen.upsert(updated_blade, write_none=True).nodes

Out[52]:

	space	instance_type	external_id	version	was_modified	last_updated_time	created_time
0	sp_wind	node	windmill:demo:write_none	2	True	2024-11-16 17:34:25.677	2024-11-16 17:34:18.147

In [53]:

retrieved_blade = pygen.blade.retrieve(new_blade.external_id)
retrieved_blade

retrieved_blade = pygen.blade.retrieve(new_blade.external_id) retrieved_blade

Out[53]:

	value
space	sp_wind
external_id	windmill:demo:write_none
data_record	{'version': 2, 'last_updated_time': 2024-11-16...
node_type	None
is_damaged	None
name	None
sensor_positions	None

Notice that the is_damaged property is now set to null as well as the name property as now all properties are explicitly set to null in the upsert call.

In [58]:

# Cleanup
pygen.delete("windmill:demo:write_none")

# Cleanup pygen.delete("windmill:demo:write_none")

Out[58]:

InstancesDeleteResult(nodes=[], edges=[])

Parameter: allow_version_increase¶

If you notice in the last examples, that when updating the blade, the version of the returning node is increasing. This is because each time we do a change to the blade node it is registered and the version is increased. You can control this behavior by setting the existing_version property in data_record of the blade node. Let's demonstrate with an example

In [60]:

from wind_turbine import data_classes as data_cls

from wind_turbine import data_classes as data_cls

In [61]:

new_blade = data_cls.BladeWrite(external_id="windmill:demo:allow_version_increase", name="Demo Blade", is_damaged=True)

new_blade = data_cls.BladeWrite(external_id="windmill:demo:allow_version_increase", name="Demo Blade", is_damaged=True)

In [62]:

pygen.upsert(new_blade).nodes

pygen.upsert(new_blade).nodes

Out[62]:

	space	instance_type	external_id	version	was_modified	last_updated_time	created_time
0	sp_wind	node	windmill:demo:allow_version_increase	1	True	2024-11-16 17:35:19.647	2024-11-16 17:35:19.647

We see that we have version 1 of the blade. If we want to avoid overwriting this blade by accident, we can set the existing_version property to lower than the version we want to avoid overwriting in the data_record of the blade node. This will be 0 in this case.

In [63]:

new_blade2 = data_cls.BladeWrite(
    external_id="windmill:demo:allow_version_increase",
    name="Demo Blade",
    is_damaged=False,
    data_record=data_cls.DataRecordWrite(existing_version=0),
)

new_blade2 = data_cls.BladeWrite( external_id="windmill:demo:allow_version_increase", name="Demo Blade", is_damaged=False, data_record=data_cls.DataRecordWrite(existing_version=0), )

In [64]:

pygen.upsert(new_blade2).nodes

pygen.upsert(new_blade2).nodes

---------------------------------------------------------------------------
CogniteAPIError                           Traceback (most recent call last)
Cell In[64], line 1
----> 1 pygen.upsert(new_blade2).nodes

File ~\Projects\internal\pygen\examples\wind_turbine\_api_client.py:101, in WindTurbineClient.upsert(self, items, replace, write_none, allow_version_increase)
     83 """Add or update (upsert) items.
     84 
     85 This method will create the nodes, edges, timeseries, files and sequences of the supplied items.
   (...)
     98 
     99 """
    100 instances = self._create_instances(items, write_none, allow_version_increase)
--> 101 result = self._client.data_modeling.instances.apply(
    102     nodes=instances.nodes,
    103     edges=instances.edges,
    104     auto_create_start_nodes=True,
    105     auto_create_end_nodes=True,
    106     replace=replace,
    107 )
    108 time_series = TimeSeriesList([])
    109 if instances.time_series:

File ~\AppData\Local\pypoetry\Cache\virtualenvs\cognite-pygen-9ATJrLbx-py3.10\lib\site-packages\cognite\client\_api\data_modeling\instances.py:1025, in InstancesAPI.apply(self, nodes, edges, auto_create_start_nodes, auto_create_end_nodes, auto_create_direct_relations, skip_on_version_conflict, replace)
   1022 edges = edges or []
   1023 edges = edges if isinstance(edges, Sequence) else [edges]
-> 1025 res = self._create_multiple(
   1026     items=cast(Sequence[WriteableCogniteResource], (*nodes, *edges)),
   1027     list_cls=_NodeOrEdgeApplyResultList,
   1028     resource_cls=_NodeOrEdgeApplyResultAdapter,  # type: ignore[type-var]
   1029     extra_body_fields=other_parameters,
   1030     input_resource_cls=_NodeOrEdgeApplyAdapter,  # type: ignore[arg-type]
   1031     executor=ConcurrencySettings.get_data_modeling_executor(),
   1032 )
   1033 return InstancesApplyResult(
   1034     nodes=NodeApplyResultList([item for item in res if isinstance(item, NodeApplyResult)]),
   1035     edges=EdgeApplyResultList([item for item in res if isinstance(item, EdgeApplyResult)]),
   1036 )

File ~\AppData\Local\pypoetry\Cache\virtualenvs\cognite-pygen-9ATJrLbx-py3.10\lib\site-packages\cognite\client\_api_client.py:949, in APIClient._create_multiple(self, items, list_cls, resource_cls, resource_path, params, headers, extra_body_fields, limit, input_resource_cls, executor, api_subversion)
    946         return dumped
    947     return el
--> 949 summary.raise_compound_exception_if_failed_tasks(
    950     task_unwrap_fn=lambda task: task[1]["items"],
    951     task_list_element_unwrap_fn=unwrap_element,
    952     str_format_element_fn=str_format_element,
    953 )
    954 created_resources = summary.joined_results(lambda res: res.json()["items"])
    956 if single_item:

File ~\AppData\Local\pypoetry\Cache\virtualenvs\cognite-pygen-9ATJrLbx-py3.10\lib\site-packages\cognite\client\utils\_concurrency.py:77, in TasksSummary.raise_compound_exception_if_failed_tasks(self, task_unwrap_fn, task_list_element_unwrap_fn, str_format_element_fn)
     75 task_lists = dict(successful=successful, failed=failed, unknown=unknown, skipped=skipped)
     76 if self.unknown_error:
---> 77     self._raise_basic_api_error(str_format_element_fn, **task_lists)
     78 if self.not_found_error:
     79     self._raise_not_found_error(str_format_element_fn, **task_lists)

File ~\AppData\Local\pypoetry\Cache\virtualenvs\cognite-pygen-9ATJrLbx-py3.10\lib\site-packages\cognite\client\utils\_concurrency.py:104, in TasksSummary._raise_basic_api_error(self, unwrap_fn, **task_lists)
    102 def _raise_basic_api_error(self, unwrap_fn: Callable, **task_lists: list) -> NoReturn:
    103     if isinstance(self.unknown_error, CogniteAPIError) and (task_lists["failed"] or task_lists["unknown"]):
--> 104         raise CogniteAPIError(
    105             message=self.unknown_error.message,
    106             code=self.unknown_error.code,
    107             x_request_id=self.unknown_error.x_request_id,
    108             missing=self.missing,
    109             duplicated=self.duplicated,
    110             extra=self.unknown_error.extra,
    111             unwrap_fn=unwrap_fn,
    112             cluster=self.cluster,
    113             **task_lists,
    114         )
    115     raise self.unknown_error

CogniteAPIError: A version conflict caused the ingest to fail. | code: 400 | X-Request-ID: 5b83a7e1-114c-9ce2-882c-792cdf913b73 | cluster: greenfield
The API Failed to process some items.
Successful (2xx): []
Unknown (5xx): []
Failed (4xx): [sp_wind:windmill:demo:allow_version_increase, ...]

This can cause problems when we want to migrate data from one project to another, or from one data model to another. It is a common pattern that we use the pygen generated SDK to retrieve from one project and then use the .as_write method to turn the retrieved read format of a node into the write format. We want to ensure we always will overwrite the nodes in the new project. Then, we can use the allow_version_increase flag to ensure that we always overwrite the all the nodes and edges we are writing will have set existing_version to None which will ensure that we always overwrite the nodes and edges.

In [65]:

retrieved_blade = pygen.blade.retrieve(new_blade.external_id)
retrieved_blade

retrieved_blade = pygen.blade.retrieve(new_blade.external_id) retrieved_blade

Out[65]:

	value
space	sp_wind
external_id	windmill:demo:allow_version_increase
data_record	{'version': 1, 'last_updated_time': 2024-11-16...
node_type	None
is_damaged	True
name	Demo Blade
sensor_positions	None

In [66]:

pygen.upsert(
    data_cls.BladeWrite(external_id="windmill:demo:allow_version_increase", name="Updated", is_damaged=True)
).nodes

pygen.upsert( data_cls.BladeWrite(external_id="windmill:demo:allow_version_increase", name="Updated", is_damaged=True) ).nodes

Out[66]:

	space	instance_type	external_id	version	was_modified	last_updated_time	created_time
0	sp_wind	node	windmill:demo:allow_version_increase	2	True	2024-11-16 17:35:43.134	2024-11-16 17:35:19.647

In [67]:

writeable_blade = retrieved_blade.as_write()

writeable_blade = retrieved_blade.as_write()

In [68]:

writeable_blade

writeable_blade

Out[68]:

	value
space	sp_wind
external_id	windmill:demo:allow_version_increase
data_record	{'existing_version': 1}
node_type	None
is_damaged	True
name	Demo Blade

In [69]:

pygen.upsert(writeable_blade).nodes

pygen.upsert(writeable_blade).nodes

---------------------------------------------------------------------------
CogniteAPIError                           Traceback (most recent call last)
Cell In[69], line 1
----> 1 pygen.upsert(writeable_blade).nodes

File ~\Projects\internal\pygen\examples\wind_turbine\_api_client.py:101, in WindTurbineClient.upsert(self, items, replace, write_none, allow_version_increase)
     83 """Add or update (upsert) items.
     84 
     85 This method will create the nodes, edges, timeseries, files and sequences of the supplied items.
   (...)
     98 
     99 """
    100 instances = self._create_instances(items, write_none, allow_version_increase)
--> 101 result = self._client.data_modeling.instances.apply(
    102     nodes=instances.nodes,
    103     edges=instances.edges,
    104     auto_create_start_nodes=True,
    105     auto_create_end_nodes=True,
    106     replace=replace,
    107 )
    108 time_series = TimeSeriesList([])
    109 if instances.time_series:

File ~\AppData\Local\pypoetry\Cache\virtualenvs\cognite-pygen-9ATJrLbx-py3.10\lib\site-packages\cognite\client\_api\data_modeling\instances.py:1025, in InstancesAPI.apply(self, nodes, edges, auto_create_start_nodes, auto_create_end_nodes, auto_create_direct_relations, skip_on_version_conflict, replace)
   1022 edges = edges or []
   1023 edges = edges if isinstance(edges, Sequence) else [edges]
-> 1025 res = self._create_multiple(
   1026     items=cast(Sequence[WriteableCogniteResource], (*nodes, *edges)),
   1027     list_cls=_NodeOrEdgeApplyResultList,
   1028     resource_cls=_NodeOrEdgeApplyResultAdapter,  # type: ignore[type-var]
   1029     extra_body_fields=other_parameters,
   1030     input_resource_cls=_NodeOrEdgeApplyAdapter,  # type: ignore[arg-type]
   1031     executor=ConcurrencySettings.get_data_modeling_executor(),
   1032 )
   1033 return InstancesApplyResult(
   1034     nodes=NodeApplyResultList([item for item in res if isinstance(item, NodeApplyResult)]),
   1035     edges=EdgeApplyResultList([item for item in res if isinstance(item, EdgeApplyResult)]),
   1036 )

File ~\AppData\Local\pypoetry\Cache\virtualenvs\cognite-pygen-9ATJrLbx-py3.10\lib\site-packages\cognite\client\_api_client.py:949, in APIClient._create_multiple(self, items, list_cls, resource_cls, resource_path, params, headers, extra_body_fields, limit, input_resource_cls, executor, api_subversion)
    946         return dumped
    947     return el
--> 949 summary.raise_compound_exception_if_failed_tasks(
    950     task_unwrap_fn=lambda task: task[1]["items"],
    951     task_list_element_unwrap_fn=unwrap_element,
    952     str_format_element_fn=str_format_element,
    953 )
    954 created_resources = summary.joined_results(lambda res: res.json()["items"])
    956 if single_item:

File ~\AppData\Local\pypoetry\Cache\virtualenvs\cognite-pygen-9ATJrLbx-py3.10\lib\site-packages\cognite\client\utils\_concurrency.py:77, in TasksSummary.raise_compound_exception_if_failed_tasks(self, task_unwrap_fn, task_list_element_unwrap_fn, str_format_element_fn)
     75 task_lists = dict(successful=successful, failed=failed, unknown=unknown, skipped=skipped)
     76 if self.unknown_error:
---> 77     self._raise_basic_api_error(str_format_element_fn, **task_lists)
     78 if self.not_found_error:
     79     self._raise_not_found_error(str_format_element_fn, **task_lists)

File ~\AppData\Local\pypoetry\Cache\virtualenvs\cognite-pygen-9ATJrLbx-py3.10\lib\site-packages\cognite\client\utils\_concurrency.py:104, in TasksSummary._raise_basic_api_error(self, unwrap_fn, **task_lists)
    102 def _raise_basic_api_error(self, unwrap_fn: Callable, **task_lists: list) -> NoReturn:
    103     if isinstance(self.unknown_error, CogniteAPIError) and (task_lists["failed"] or task_lists["unknown"]):
--> 104         raise CogniteAPIError(
    105             message=self.unknown_error.message,
    106             code=self.unknown_error.code,
    107             x_request_id=self.unknown_error.x_request_id,
    108             missing=self.missing,
    109             duplicated=self.duplicated,
    110             extra=self.unknown_error.extra,
    111             unwrap_fn=unwrap_fn,
    112             cluster=self.cluster,
    113             **task_lists,
    114         )
    115     raise self.unknown_error

CogniteAPIError: A version conflict caused the ingest to fail. | code: 400 | X-Request-ID: 0d552305-b242-94ff-a8a0-275d2c2bb033 | cluster: greenfield
The API Failed to process some items.
Successful (2xx): []
Unknown (5xx): []
Failed (4xx): [sp_wind:windmill:demo:allow_version_increase, ...]

In [70]:

pygen.upsert(writeable_blade, allow_version_increase=True).nodes

pygen.upsert(writeable_blade, allow_version_increase=True).nodes

Out[70]:

	space	instance_type	external_id	version	was_modified	last_updated_time	created_time
0	sp_wind	node	windmill:demo:allow_version_increase	3	True	2024-11-16 17:35:57.577	2024-11-16 17:35:19.647

We see that the allow_version_increase flag ensures that we always overwrite the nodes and edges.

In [71]:

# Cleanup
pygen.delete("windmill:demo:allow_version_increase")

# Cleanup pygen.delete("windmill:demo:allow_version_increase")

Out[71]:

InstancesDeleteResult(nodes=[NodeId(space='sp_wind', external_id='windmill:demo:allow_version_increase')], edges=[])

Creating from JSON Format¶

See the quick start guide data population for an example of creating instances from JSON.

Deleting Instances¶

You can delete by passing and external ID or an Item or a sequence of external id or items to the delete method.

We can delete the newly created windmill

In [72]:

pygen.wind_turbine.list()

pygen.wind_turbine.list()

Out[72]:

	space	external_id	capacity	name	blades	datasheets	nacelle	rotor	windfarm	data_record
0	sp_wind	hornsea_1_mill_3	7.0	hornsea_1_mill_3	[hornsea_1_mill_3_blade_A, hornsea_1_mill_3_bl...	[windmill_schematics]	hornsea_1_mill_3_nacelle	hornsea_1_mill_3_rotor	Hornsea 1	{'version': 4, 'last_updated_time': 2024-11-16...
1	sp_wind	hornsea_1_mill_2	7.0	hornsea_1_mill_2	[hornsea_1_mill_2_blade_B, hornsea_1_mill_2_bl...	[windmill_schematics]	hornsea_1_mill_2_nacelle	hornsea_1_mill_2_rotor	Hornsea 1	{'version': 4, 'last_updated_time': 2024-11-16...
2	sp_wind	hornsea_1_mill_1	7.0	hornsea_1_mill_1	[hornsea_1_mill_1_blade_A, hornsea_1_mill_1_bl...	[windmill_schematics]	hornsea_1_mill_1_nacelle	hornsea_1_mill_1_rotor	Hornsea 1	{'version': 4, 'last_updated_time': 2024-11-16...
3	sp_wind	hornsea_1_mill_4	7.0	hornsea_1_mill_4	[hornsea_1_mill_4_blade_C, hornsea_1_mill_4_bl...	[windmill_schematics]	hornsea_1_mill_4_nacelle	hornsea_1_mill_4_rotor	Hornsea 1	{'version': 4, 'last_updated_time': 2024-11-16...
4	sp_wind	hornsea_1_mill_5	7.0	hornsea_1_mill_5	[hornsea_1_mill_5_blade_B, hornsea_1_mill_5_bl...	[windmill_schematics]	hornsea_1_mill_5_nacelle	hornsea_1_mill_5_rotor	Hornsea 1	{'version': 4, 'last_updated_time': 2024-11-16...
5	sp_wind	windmill:demo	10.0	Windmill ATH	[windmill:demo:blade1, windmill:demo:blade2, w...	[]	windmill:demo:nacelle	windmill:demo:rotor	Fornebu	{'version': 1, 'last_updated_time': 2024-11-16...

Same as .upsert, the delete method is located on the pygen and not the API class.

In [73]:

deleted = pygen.delete(new_turbine)

deleted = pygen.delete(new_turbine)

In [74]:

len(deleted.nodes), len(deleted.edges)

len(deleted.nodes), len(deleted.edges)

Out[74]:

(4, 0)

After the delete call the new windmill is gone

In [75]:

pygen.wind_turbine.list()

pygen.wind_turbine.list()

Out[75]:

	space	external_id	capacity	name	blades	datasheets	nacelle	rotor	windfarm	data_record
0	sp_wind	hornsea_1_mill_3	7.0	hornsea_1_mill_3	[hornsea_1_mill_3_blade_A, hornsea_1_mill_3_bl...	[windmill_schematics]	hornsea_1_mill_3_nacelle	hornsea_1_mill_3_rotor	Hornsea 1	{'version': 4, 'last_updated_time': 2024-11-16...
1	sp_wind	hornsea_1_mill_2	7.0	hornsea_1_mill_2	[hornsea_1_mill_2_blade_B, hornsea_1_mill_2_bl...	[windmill_schematics]	hornsea_1_mill_2_nacelle	hornsea_1_mill_2_rotor	Hornsea 1	{'version': 4, 'last_updated_time': 2024-11-16...
2	sp_wind	hornsea_1_mill_1	7.0	hornsea_1_mill_1	[hornsea_1_mill_1_blade_A, hornsea_1_mill_1_bl...	[windmill_schematics]	hornsea_1_mill_1_nacelle	hornsea_1_mill_1_rotor	Hornsea 1	{'version': 4, 'last_updated_time': 2024-11-16...
3	sp_wind	hornsea_1_mill_4	7.0	hornsea_1_mill_4	[hornsea_1_mill_4_blade_C, hornsea_1_mill_4_bl...	[windmill_schematics]	hornsea_1_mill_4_nacelle	hornsea_1_mill_4_rotor	Hornsea 1	{'version': 4, 'last_updated_time': 2024-11-16...
4	sp_wind	hornsea_1_mill_5	7.0	hornsea_1_mill_5	[hornsea_1_mill_5_blade_B, hornsea_1_mill_5_bl...	[windmill_schematics]	hornsea_1_mill_5_nacelle	hornsea_1_mill_5_rotor	Hornsea 1	{'version': 4, 'last_updated_time': 2024-11-16...

In addition, since we passed in the entire object and not just an external ID, all the edges and nodes connected to the windmill:demo was also deleted.

In [ ]: