Skip to content

4473 expand reminder date possibilities #5190

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 63 commits into
base: main
Choose a base branch
from

Conversation

Benjamin-Couey
Copy link
Contributor

Resolves #4473

Description

This PR is built off of the work done by jlandiseigsti.

This PR expands the _deadline_day_fields.html.erb partial to include options to configure:

  • The monthly frequency of reminders
  • The date of the month the reminder will be sent
  • The nth weekday of the month the reminder will be sent

This definition is then saved as an ICAL format string to the new reminder_schedule field of the Organization or PartnerGroup models which replaces the old reminder_day field. The ice_cube gem is used to convert to and from the ICAL format schedule.

The form is reactive, and will show users a preview of when the next reminder/deadline will be based on their selection as well as warns them if their choices violates a constraint. This is implemented using a Stimulus controller reworked from the prior deadline_day_pickers.js.

This PR expands the Deadlinable ActiveSupport::Concern, to contain the logic for validating the fields of the _deadline_day_fields.html.erb, converting the values to ICAL format, etc.

This PR modifies the FetchPartnersToRemindNowService to consider the Organization and PartnerGroup reminder_schedule when determining if a partner should be emailed.

The documentation in the user guide has been updated and expanded to more thoroughly explain these features. Included is the excellent flowchart which clief created.

This PR adds the following dependencies:

In my investigation of the rrule library, I found the similar rschedule library. While I went with rrule because it was older and had more widespread usage, if some issue with rrule comes up rschedule seems like a good first place to look next.

It was considered changing how the Partner.send_reminders field worked, as it's strange that it matters for Organization level schedules, but not for PartnerGroup level schedules. I am still of the opinion this is something that should be fixed, but when discussed with @cielf it was determined it could be saved for another PR.

Similarly, it was considered reworking deadline_day to be similar to reminder_schedule and permit deadlines to be defined as "The Nth Weekday of the month". Since this wasn't specifically requested, it was determined it could be saved for another PR.

The deadline_day field is constrained to be between 1 and 28. For some reason, this constraint is enforced in the db/schema.rb for PartnerGroup.deadline_day but not Organization.deadline_day. I refrained from modifying either definition in the schema because the intention was unclear to me, and I think it possible deadline_day will be reworked as described above anyway.

Type of change

  • New feature (non-breaking change which adds functionality)
  • Documentation update

How Has This Been Tested?

I manually tested that the form and it's dynamic elements worked as expected.

Multiple tests were added to spec/models/concerns/deadlinable_spec.rb to verify the various new functions added.

Tests validating the constraints on reminder_schedule and deadline_day fields in spec/models/organization_spec.rb and spec/models/partner_group_spec.rb were removed, as they were redundant with the similar tests added to deadlinable_spec.rb and Deadlinable is the place where those constraints are validated.

A test was added to spec/services/deadline_service_spec.rb to verify that PartnerGroup.reminder_schedule is used in place of Organization.reminder_schedule when available.

spec/services/partners/fetch_partners_to_remind_now_service_spec.rb was updated to use the new reminder_schedule format, to test different types of schedules, and to ensure Partner.send_reminders worked as it currently does.

Multiple tests were added in spec/support/deadline_day_fields_shared_example.rb which verify the use of the _deadline_day_fields.html.erb form in the admin, organization, and partner group contexts. Included was a test to verify that the next reminder and deadline dates calculated by the frontend match the calculations on the backend.

Included in jlandiseigsti's work were tests added to spec/system/organization_system_spec.rb verifying assorted viewing and editing workflows that were not directly related to the issue. They seemed like fine tests and it didn't seem worth the effor to make a separate PR for them so they were kept around.

Screenshots

The new deadline day form, showing the calculated reminder and deadline dates calculated for a schedule by a day of the month

DeadlineDayFormDayOfMonth

The new deadline day form, showing the calculated reminder and deadline dates calculated for a schedule by a weekday

DeadlineDayFormNthWeekday

The new deadline day form, showing the deadline date calculated to be next month since otherwise it would be prior to the reminder

DeadlineDayFormNextMonth

The new deadline day form, showing a warning when the entered values are outside bounds

DeadlineDayFormOutsideBounds

The new deadline day form, showing a warning when reminder and deadline day are the same

DeadlineDayFormCannotBeSame

The organization's details, updated with a new schedule

UpdatedOrganization

jlandiseigsti and others added 30 commits April 7, 2025 14:38
Via db migrations, replaces the simple integer `reminder_day` with the more complex
`reminder_schedule` which is an ical string that can be parsed to a repeating Schedule class.
Adjusts the logic in the `fetch_partners_to_reminder_now_service` to use the repeating schedule.
Updates related tests.
Builds an ActiveModel, ReminderSchedule, which takes the necessary information
and turns it into an IceCubeSchedule, which is what ultimate get saved in the db.

Builds the form for this reminder schedule. Still needs to conditionally show or hide
sections depending on if the user wants date or day of the week.
Now either while creating new Organization/Partner Groups or editing them,
all the transfers from Reminder Date to Reminder Day have been made.
This uses css for ease and accessibility.
 Adds a number of refactors, including moving all the create_schedule logic
 to the deadlinable helper, and creates unit and system tests for the new functionality.
Reintroduces the functionality of having warnings for having the reminder
date the same as the deadline date.
We don't want to create a new schedule if nothing has changed,
as it will reset the start date and potentially mess with those
who have every 2/3/etc months reminders.
All reminders should be Monthly, so every N Months is removed.

Also allows users to select "Last" as an option.
Rather than "date" and "week_day", we are now using "day_of_month" and
"day_of_week" for clarity.
There was a bug in which if the reminder day of the month was the same
as the deadline day, it would dislplay the error message even if the user
changed to the day of the week option. Now the error message is hidden if
the user is selecting the day of the week option.
…ails page and verify new reminder schedule interface
…o new stimulus controller, replaced conditional SASS rule with JS implementation for sake of readability/maintainability
…yOfMonth and deadlineDay fields, as exact remidner date will shift when using byDayOfWeek
…lic since they don't have side effects, tweaked should_update_reminder_schedule
…s part of a group with reminders, it still receives a reminder
@Benjamin-Couey
Copy link
Contributor Author

@cielf while updating stuff so an entered start date remains persistent across page loads, some weird behavior I hadn't anticipated came up that I wanted your opinion on.

Because the start date field auto fills with today in the form, the first time anyone saves the org details form that start date will be saved. While technically consistent with what you described, it seems unintuitive that making no changes in a form but clicking the save button will create persistent data for the object.

How do you want to handle this situation?

@cielf
Copy link
Collaborator

cielf commented May 21, 2025

Well, I suppose we could save it only if the other date info is filled in.

That would mean that for the people who don't currently have reminder dates, if they ever decide to, they'd be working with a default of 'today' when they do set somehing up. The people who already have reminder dates, it shouldn't be a big issue for.

Does that make sense to you?

@Benjamin-Couey
Copy link
Contributor Author

Well, I suppose we could save it only if the other date info is filled in.

That would mean that for the people who don't currently have reminder dates, if they ever decide to, they'd be working with a default of 'today' when they do set somehing up. The people who already have reminder dates, it shouldn't be a big issue for.

Does that make sense to you?

That make sense, thank you.

@cielf
Copy link
Collaborator

cielf commented May 22, 2025

FWIW, Probably should put a comment to explain why in the code -- this is sufficiently unusual that it won't be intuitively obvious to future coders.

@Benjamin-Couey
Copy link
Contributor Author

@cielf Alright, pushed some commits addressing things that came up in the last office hours. Namely:

  • The start date field now comes after the monthly frequency field and is labeled "Reminder start being sent after:"
  • The messages that appear as the user fills out the form now read "Your next reminder date is [reminder date]."
  • The dates calculated on the front end now take into account both the start date of the reminder schedule, as well as the current date. For example, if today is the 22nd, the reminder is on the 21st, and the start date is the 20th, the interface will tell the user their next reminder date is June 21st.
  • As a consequence of this, the interface won't tell the user that their next reminder is today. Given the inconsistency of reminders firing on the day they were set up that we discussed in the office hours, this seems desirable. Once we're happy with the interface, I'll make a special note about this in the user guide).

@cielf
Copy link
Collaborator

cielf commented May 23, 2025

@Benjamin-Couey Tiny change #1: The label "Reminder start being sent after:" Should be "Reminders start being sent after"

@cielf
Copy link
Collaborator

cielf commented May 23, 2025

Otherwise the bank-facing functionality looks good. Manually testing the sending would be difficult.

I think we can send this to @dorner for all the techical advice. (And I'll rely on him to determine that the testing of the outbound is sufficient).

@cielf cielf requested a review from dorner May 23, 2025 15:10
Copy link
Collaborator

@dorner dorner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a huge project - thanks so much for taking it on! I'd like to tweak the direction though.

:repackage_essentials, :distribute_monthly,
:ndbn_member_id, :enable_child_based_requests,
:enable_individual_requests, :enable_quantity_based_requests,
:ytd_on_distribution_printout, :one_step_partner_invite,
:hide_value_columns_on_receipt, :hide_package_column_on_receipt,
:signature_for_distribution_pdf, :receive_email_on_requests,
:start_date, :by_month_or_week, :day_of_month, :day_of_week, :every_nth_day,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like this list of symbols should be stored in a constant somewhere we can reference. If we ever want to improve this, we'll have to remember to do this everywhere.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So... I get why this was originally a concern, but this has ballooned way out of proportion for something that should be tacked onto a model.

I'd prefer this be made a service class which can use ActiveModel validations. The individual model can be associated with that class (e.g. Schedule) and we can leave all the functions and validations there without polluting the models that use it.

So you could have a method in Organization that returns a Schedule:

def deadline
  Schedule.new(self.deadline_schedule)
end

and then you could call the relevant methods on that object, which is a PORO and not an ActiveRecord.

.where.not(partner_groups: {deadline_day: nil})
.where.not(status: deactivated_status)

# where partner groups have reminder schedule match
filtered_partner_groups = partners_with_group_reminders.select do |partner|
sched = IceCube::Schedule.from_ical partner.partner_group.reminder_schedule
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think any object outside of the DeadlineSchedule should reach into Ical. This is another good reason for the separation. We can expose this as a method on DeadlineSchedule, e.g. def self.occurs_on?(deadlineable, day)

@@ -0,0 +1,6 @@
class RemoveReminderDayFromOrganizations < ActiveRecord::Migration[7.1]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should leave this removal out until we're sure the data has been copied over correctly? We can do a fast-follow PR to remove it.

@@ -44,20 +43,22 @@
skip_items { false }
end

recurrence_schedule = IceCube::Schedule.new
recurrence_schedule.add_recurrence_rule IceCube::Rule.monthly(1).day_of_month(10)
recurrence_schedule_ical = recurrence_schedule.to_ical
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Being able to limit the "schedule" part of this logic to a single place would also be a big win IMO. Then all you need to do is make sure that the organization's reminder_schedule method returns a Schedule with the right data, and you're good.

@Benjamin-Couey
Copy link
Contributor Author

@dorner regarding your comments.

I think I get the idea and I'll start working on reworking what I have. There is one point I'd like some clarification.

Then all you need to do is make sure that the organization's reminder_schedule method returns a Schedule with the right data, and you're good.

reminder_schedule is currently the name of the field that stores the schedule information as an ICal string. Am I correct in assuming that's still how we want to store the data in the database, we'd just like there to also be a method in the service class that spits out a service object from an ActiveRecord?

@dorner
Copy link
Collaborator

dorner commented May 30, 2025

Yeah. Probably the cleanest API would be something like:

# Model
def reminder_schedule
  Schedule.new(self.reminder_schedule_definition) # maybe rename the column to this
end

Then we don't have to assume anything about the underlying database schema.

…der_day field, and added first pass at ReminderScheduleService [skip ci]
@Benjamin-Couey
Copy link
Contributor Author

@dorner Alright, the most recent commit contains my first pass on the new service object you described. I still need to hook it up everywhere besides the Organization page and update the tests, but I wanted to get your opinion on the overall architecture before I went to far.

As an overview, ReminderScheduleService is a class that contains all the IceCube logic and all the fields and validation needed to make a reminder schedule, and methods to spit out the schedule description and Ical representation.

This service is associated to models with the ReminderScheduleable concern which adds getters and setters so the ReminderScheduleService fields can be set through the model and callbacks to update/read the reminder_schedule_definition.

Copy link
Collaborator

@dorner dorner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting a lot closer! I like the initial approach, but have some suggestions.


# For now assume that you won't be setting individual fields.
# You'll only be updating the ReminderScheduleService via saving an object with params.
def every_nth_month = reminder_schedule&.every_nth_month
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this? I'd rather just call reminder_schedule.foo methods directly than obscure it through the concern.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason I added these getters and setters was so we could just rely on Rail's update function to set the ReminderScheduleService when called on a model associated with the ReminderScheduleService.

This still seems like a valuable feature to me, as it helps keep the implementation of the actual frontend form and the handling of the parameters simpler.

after_save :reset_reminder_schedule_service

validate :reminder_schedule_is_valid?
validates :deadline_day, numericality: {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't seem to have anything to do with the reminder schedule?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fair. The Deadlineable concern also validated the deadline_day field, presumably because the day_of_month needs to be checked against the deadline_day. That said, it probably makes more sense for the validation for a model's field to be in the model itself.

@@ -0,0 +1,92 @@
module ReminderScheduleable
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd question strongly whether we need yet another concern. I think if you look at how things are called, we're just saving a handful of lines of code. This concern presupposes things about the underlying model (e.g. what the column that holds the reminder schedule is called).

See what happens if you just reference org.reminder_schedule.whatever instead of trying to go directly on the model. My hunch is it would be more or less the same as this without a lot of the extra redirection.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is certainly an issue that the concern makes assumptions about the associated model. I'm still of the opinion that there is value in the getters and setters, but I will give this a try.


include ActiveModel::Validations

validate :parent_object_has_deadline_day_field?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this service should know anything about the parent object. It should be strict input / output - it's initialized with a string or parameters, and you can query its fields and output a string or parameters.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fair.


# TODO: Consider reworking this to validate the IceCube schedule that gets generated, so it checks both day_of_month and day_of_week
# schedules
def deadline_not_on_reminder_date?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reverses the correct flow - the parent object should have a validation that does if self.reminder_schedule.day_of_month.to_i != self.deadline_day.to_i

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense if the goal is for ReminderScheduleService to be a strict input/output interface.

@@ -174,7 +174,7 @@
<h6 class="font-weight-bold">Default reminder day (day of month an email reminder to submit Requests is sent to Partners)</h6>
<p>
<%= fa_icon "calendar" %>
<%= @organization.reminder_schedule.blank? ? 'Not defined' : @organization.show_description(@organization.reminder_schedule) %>
<%= @organization.reminder_schedule.blank? ? 'Not defined' : @organization.reminder_schedule.show_description %>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@organization.reminder_schedule&.show_description || 'Not defined'

recurrence_schedule = IceCube::Schedule.new
recurrence_schedule.add_recurrence_rule IceCube::Rule.monthly(1).day_of_month(10)
recurrence_schedule_ical = recurrence_schedule.to_ical
reminder_schedule_definition = ReminderScheduleService.new({
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this supposed to be a string? Not a ReminderScheduleService?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are correct. It gets converted to an Ical string later but as it's written now is just confusing. Something like:

reminder_schedule_definition { ReminderScheduleService.new({
  every_nth_month: "1",
  by_month_or_week: "day_of_month",
  day_of_month: 10
}).to_ical }

Correct me if I'm wrong, but I do still think we should be using ReminderScheduleService her to go from a set of parameters to an Ical string.

@Benjamin-Couey
Copy link
Contributor Author

@dorner

I have some questions about your overall vision for the ReminderScheduleService, namely how data flows from fields in the form, to parameters in a controller action, to the fields of a ReminderScheduleService object when we remove the getters and setters that make the fields of ReminderScheduleService virtual fields of the associated ActiveRecord object.

If we look at the form, we see that it's currently using the simple form gem. The form is reused in multiple views where a simple form object (f) is initialized for a specific active record model (for example, this partner group view or this organization view).

If the associated ActiveRecord object (Organization or PartnerGroup), doesn't have the attributes associated with these fields, this whole form would need to be reworked. At the moment, I can't see a super straightforward way to do that, though perhaps I'm missing something.

Once the parameters have been passed to the controller, we then need to either create a ReminderScheduleService object with them or update an existing one. Again, giving the ActiveRecord object the virtual fields helps here, letting the existing assignment of parameters to the object also update the ReminderScheduleService. The alternative I can think of is to manually build the ReminderScheduleService from the params, which is admittedly not horrible but still not as clean.

How do you envision handling the data flow I described?

@dorner
Copy link
Collaborator

dorner commented Jun 11, 2025

You should be able to do a fields_for :schedule_reminder inside the form. You might need to do a bit of twiddling, but I don't think the form requires an ActiveRecord necessarily, so the accessors should work well enough to print the correct HTML for the form. This would give you a nested part of the form that can be taken out in the controller to pass into the ReminderScheduleService.

@Benjamin-Couey
Copy link
Contributor Author

@dorner Alright, third time's the charm. The recent commit contains my attempt removing the concern as you described. As before, it still needs to be hooked up to the PartnerGroup and the tests need updating, but I wanted to get your opinion on the architecture.

As on overview:
The Organization is now responsible for all validations relating to the deadline_day field. The ReminderScheduleService no longer makes any assumptions nor holds a reference to it's parent object.

The _deadline_day_fields.html.erb form now creates a nested form for the parent object's ReminderScheduleService object's fields. deadline_day is still associated with the parent object.

A separate reminder_schedule_params was added to the OrganizationsController controller. Prior to an Organization being updated, these params will be used to update the org's ReminderScheduleService object.

Copy link
Collaborator

@dorner dorner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor suggestion, but otherwise I like the approach!

@@ -70,6 +70,12 @@ def self.from_ical(ical)
})
end

def assign_attributes(attrs)
attrs.each_pair do |attr, value|
instance_variable_set("@#{attr}", value)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather this look a bit cleaner...

def []=(key, val)
  self.send("#{key}=", val)
end

def assign_attributes(attrs)
  attrs.each { |key, val| self[key] = val }
end

@Benjamin-Couey
Copy link
Contributor Author

One minor suggestion, but otherwise I like the approach!

Good to hear. I'll make the tweak you suggested and then get started hooking everything up and updating the tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Expand possible dates for reminder emails
4 participants