On Mon, Jun 27, 2022 at 07:06:43AM -0000, Christian Ehrhardt wrote:
> I guess the underlying case here is that we want "some kind of monitoring" to alert us of failed imports, but do not insist too hard which solution that is.
>
> Is there already something new present and working that will replace the old mailing feature?
Since then we now have the tracking system. We have a database tracking
progress and can list errored imports from the database. So we aren't
getting directly notified, but can generate a report of everything that
needs retrying, for example.
In practice, this emailing code was useful when we had many failures for
various reasons and were struggling to keep track of them. Now, the
majority of those issues are fixed, the remaining failures are mostly
transient and I don't think that receiving emails for them invidiually
would be useful any more. Maybe we should have some automatic retry
system hit failures first, but we don't currently have that.
I spoke to Andreas about future design for observability/monitoring and
I think the conclusion was that InfluxDB probably made sense to log
"events" such as "package A succeeded; processing time X" and "package B
failed". But no, that's not there today, and it's not currently a
priority to implement.
> I know it was off recently, but dropping it entirely from the codebase without a successor feels odd indeed.
I'm surprised at this reaction because I don't see this as a decision
that's being made now. The code has already been "skeleton" for a long
time.
It is already effectively dropped - it isn't connected to any live code.
It's de-facto not being maintained _because_ it isn't connected to
anything so doesn't get adjusted as the rest of the code changes.
Reconnecting it would involve some work because the paradigm has
changed. We no longer have a set of failures that happen in sequence
that we then batch the results email out.
Removing it is merely cleaning up the source tree for a change that
already took place. If we want to bring it back to live status later, it
will still be in VCS. It makes little difference whether it is in the
source tree or in VCS history except that if it stays I have to keep
half-maintaining it when refactoring.
On Mon, Jun 27, 2022 at 07:06:43AM -0000, Christian Ehrhardt wrote:
> I guess the underlying case here is that we want "some kind of monitoring" to alert us of failed imports, but do not insist too hard which solution that is.
>
> Is there already something new present and working that will replace the old mailing feature?
Since then we now have the tracking system. We have a database tracking
progress and can list errored imports from the database. So we aren't
getting directly notified, but can generate a report of everything that
needs retrying, for example.
In practice, this emailing code was useful when we had many failures for
various reasons and were struggling to keep track of them. Now, the
majority of those issues are fixed, the remaining failures are mostly
transient and I don't think that receiving emails for them invidiually
would be useful any more. Maybe we should have some automatic retry
system hit failures first, but we don't currently have that.
I spoke to Andreas about future design for observability/ monitoring and
I think the conclusion was that InfluxDB probably made sense to log
"events" such as "package A succeeded; processing time X" and "package B
failed". But no, that's not there today, and it's not currently a
priority to implement.
> I know it was off recently, but dropping it entirely from the codebase without a successor feels odd indeed.
I'm surprised at this reaction because I don't see this as a decision
that's being made now. The code has already been "skeleton" for a long
time.
It is already effectively dropped - it isn't connected to any live code.
It's de-facto not being maintained _because_ it isn't connected to
anything so doesn't get adjusted as the rest of the code changes.
Reconnecting it would involve some work because the paradigm has
changed. We no longer have a set of failures that happen in sequence
that we then batch the results email out.
Removing it is merely cleaning up the source tree for a change that
already took place. If we want to bring it back to live status later, it
will still be in VCS. It makes little difference whether it is in the
source tree or in VCS history except that if it stays I have to keep
half-maintaining it when refactoring.
That's the cost. What would be the benefit?