plotstream and streamlist ( with current plotstream in it ) can be out of sync

[stephanbracke]

While cleaning a baseline plot by removing the flagged points, it will update the current plotstream with nan on the place of the flags.

magpy/magpy/gui/magpy_gui.py

Line 5962 in ae91462

self.plotstream = self.plotstream.remove_flagged()

However on that moment there is also self.streamlist and on index 0 (if it is the only loaded data) you find the same stream but without the flag removal.
When we now click on fit we see a fit happening by using the plotstream (with flag removal )

magpy/magpy/gui/magpy_gui.py

Lines 4515 to 4519 in ae91462

	func = self.plotstream.fit(keys=keys,
	fitfunc=params['fitfunc'],
	fitdegree=params['degree'], knotstep=params['knots'],
	starttime=params['starttime'],
	endtime=params['endtime'])

This one is ok.
If we now load a data file, the baseline button becomes available. By clicking on this button we will arrive in following code

magpy/magpy/gui/magpy_gui.py

Line 4913 in ae91462

baselinefunc = self.plotstream.baseline(absstream,fitfunc=fitfunc, knotstep=float(knotstep), fitdegree=int(degree), startabs=starttime, endabs=endtime)

With abstream being

magpy/magpy/gui/magpy_gui.py

Line 4893 in ae91462

absstream = self.streamlist[int(basedict.get('streamidx'))]

and later on this abstream = bas is used to fit

magpy/magpy/stream.py

Line 2451 in ae91462

func = bas.fit(keys,fitfunc=fitfunc,fitdegree=fitdegree,knotstep=knotstep)

This abstream is however the stream out of the self.streamlist that hasn't taken into account the dropped flags and will result into a faulty baseline fit.
As a work around we saved the cleaned plotstream into a magpy cdf and after closing magpy and reloading this cdf file the self.streamlist will be correct and we can use a correct fit on the data.
As another test to be sure it was caused by these streams not being insync I quick/dirty inserted the line just after line 5962 plotstream.remove_flagged()

        self.streamlist[self.currentstreamindex] = self.plotstream

If we are sure that the currentstreamindex always correspond with the current plotstream we could accept to solve it this way but for this I need to investigate further.
However we need to make sure when plotstream is changed and it has a possible use later on afterwards the stream that correponds in the streamlist should also be synced with this plotstream

[leonro]

Have you tried to add the current working state to the memory. i.e. add a new instance to self.streamlist?
This should be done directly after the flags have been removed.

Just checked that: at the moment the baseline dictionary is only updated when initially loading the file, not when updating baseline data using "add to working state". I will work on a solution...

[leonro]

Each instance of basevalue data (as soon as added to the memory) will now get an index for selection. Solved in 3fa7476

[stephanbracke]

This is still a problem just tested it on branch gui-fixes/linux

[stephanbracke]

Have you tried to add the current working state to the memory. i.e. add a new instance to self.streamlist? This should be done directly after the flags have been removed.

Just checked that: at the moment the baseline dictionary is only updated when initially loading the file, not when updating baseline data using "add to working state". I will work on a solution...

I just tried when you push on the add current working state to memory it will work but what is confusing to the user is that you can push on the baseline button while you forgot to push to the add to current ... then you get a baseline fit without taken into account the dropped flags

[stephanbracke]

I think I have a simple solution to solve this issue
For the moment you have in the magpy_gui there is an InitialRead Method
Here we have

magpy/magpy/gui/magpy_gui.py

Line 1875 in ae91462

self.plotstream = self.stream.copy()

If you remove the copy the plotstream and streamlist will be aligned so everything you do on the plotstream will be reflected in the streams as well

[leonro]

If I remove the copy() method here then I will loose the functionality of the "restore" button on the Data panel. This "restore" possibility however is something which at least I use quite frequently. I personally would prefer to keep it as suggested above which however requires the " add ... to memory". Eventually one could add some "don't forget to save" notification box, if a data set has been flagged etc and the user wants to open a new data set.

[stephanbracke]

Ok,
I didn't know it was linked with the restore.
I investigated the code further and their are other ways of resolving it

• we really want users to add to memory before using it
• we make sure that when a new dataset is loaded we update the streamlist

Second one is the most easy for the user new dataset will be loaded but before that we update the streams with the cleaned plotstream
To do that we go to the method InitialRead

we check if streamlist is not empty and if so we update the streamlist with the latest plotstream here is it situated

magpy/magpy/gui/magpy_gui.py

Lines 1868 to 1877 in 6a560b1

	def InitialRead(self,stream):
	"""
	DESCRIPTION
	Backups stream content and adds current strem and header info to streamlist and headerlist.
	Creates plotstream copy and stores pointer towards lists.
	Checks whether ndarray is resent and whether data is present at all
	Eventually extracts flaglist
	"""
	if not len(stream.ndarray[0]) > 0:

you can insert just as first lines

       if len(self.streamlist) > 0:
            self.streamlist[self.currentstreamindex] = self.plotstream

in method InitialRead

If you however don't want this but you prefer that users always push on the add to memory stream button you could eliminated following lines

magpy/magpy/gui/magpy_gui.py

Lines 1661 to 1671 in 6a560b1

	if formattype == 'MagPyDI' or contenttype.startswith('MagPyDI'):
	if not sensorid and not dataid:
	basename = self.menu_p.str_page.fileTextCtrl.GetValue()
	elif dataid:
	basename = dataid
	else:
	basename = sensorid
	# Obtain the correct stream idx function
	streamidx = self.currentstreamindex
	self.append_baseline(mintime,maxtime,basename,streamidx)

Just comment them and the baseline with the current plot will not be added, so button on baseline will not be available when loading a new plot stream