WIP: Caching spacy doc #93

sarthakbabbar3 · 2023-06-16T15:08:25Z

No description provided.

…. We had broken the connection between doc and student with the reorder and were pulling in errors from inactive students. This presents a fix for both things.

…t we had. We had broken the connection between doc and student with the reorder and were pulling in errors from inactive students. This presents a fix for both things." This reverts commit 9f96e25.

Add menu dropdown and status button

Lastdocfix

…_observer into lastdocfix

… contents to writing analysis.

Lastdocfix

… APIs

…library

…rver

…tarting to move rosters to use new API

- Cleaned up lots of duplicated code - Added child support to possible selectors - Removed text from settings option, focusing on highlighting, we can revisit text as a component later - modularized pieces of dashboard

* Fixing the manifest issue for permissions. * Fixing manifest to reflect new version and to remove permissions that we do not make use of.

…erver Dealing with existing conflicts dfuring merge.

* Caching Version 0 Initial Commit * Caching Version 0 Initial Commit With Fixes * Caching Version 0 Initial Commit With Fixes * Caching Version 0 Initial Commit With Fixes * Refactoring Caching Code * Modularize Caching function, add helper functions to util.py * Improve modularization and reducing dirty reads. * Renamed functions & variables, added parameter definitions * Reduce gap between cache read and write * Remove blank lines --------- Co-authored-by: Bradley Erickson <[email protected]>

Merge Master into Caching

bradley-erickson · 2023-06-23T15:19:00Z

modules/writing_observer/writing_observer/awe_nlp.py



-async def process_texts_serial(texts, options=None):
+async def process_texts_serial(texts, doc=None, options=None):


Currently unsued method. This will be removed in the future with the communication protocol.

This can stay as is.

bradley-erickson · 2023-06-23T15:19:17Z

modules/writing_observer/writing_observer/awe_nlp.py



-async def process_texts_parallel(texts, options=None):
+async def process_texts_parallel(texts, doc=None, options=None):


Currently unsued method. This will be removed in the future with the communication protocol.

This can stay as is.

bradley-erickson · 2023-06-23T15:21:34Z

modules/writing_observer/writing_observer/awe_nlp.py

    :param writing: The writing data.
    :return: The updated writing data.
    """
+    # Calculate the features required and prepare the temporary cache dict


compute_nlp_features_needed() add_to_our_needed_nlp_features_queue() while await_running_processes(): random_delay compute_nlp_features_we_need_and_queue_needs() update_cache_to_tell_it_were_running_and_what_were_running() run_needed_features() update_cache() return

prior person wanted: A and D
brad: I want B and C
sarthak: want C and D and E

Brad is running on the system running features B and C.
Then you make your request for C and D and E
compute_nlp_features_needed - return C, E since it found D from the prior
we add C, E to the queue of features needing to be run - add_to_our_needed_nlp_features_queue

Wait for any blocking features (i.e. brad running B and C) . 0-100ms

compute_nlp_features_we_andthe_queue_needs - Only returns E since C is covered from Brad's run
We start running E, telling the cache we are running E and then actually running E.

Update with E and return the items we need,

* initial new layout for askgpt dashbaord * Documented code

JohnDamilola and others added 30 commits December 23, 2022 11:24

Add settings to dropdown menu

092fb0e

Update top-right button styles

6f29408

Making fixes across the system to switch camel to underscore case.

2818dae

Putting in place a fix for the document alignment problem that we had…

9f96e25

…. We had broken the connection between doc and student with the reorder and were pulling in errors from inactive students. This presents a fix for both things.

Revert "Putting in place a fix for the document alignment problem tha…

810a249

…t we had. We had broken the connection between doc and student with the reorder and were pulling in errors from inactive students. This presents a fix for both things." This reverts commit 9f96e25.

Adding fix for last-doc align via the event-wrapper.

4749a25

Adding minor fix to wrapper which should address some basic errors.

ef87b30

Merge pull request #41 from ArgLab/dami-top-right-btn

4cffc4c

Add menu dropdown and status button

Removing some unnecessary info.

239e059

Making a doc cleanup.

6945d28

Merge pull request #42 from ArgLab/lastdocfix

955a976

Lastdocfix

Adding note on bcrypt to readme.

406ea86

Updating writing analysis to better deal with event types.

9f44e93

Merge branch 'lastdocfix' of https://github.com/ArgLab/ArgLab_writing…

edfaab8

…_observer into lastdocfix

Updating to deal with unrecognized last docs and moving event wrapper…

5951e93

… contents to writing analysis.

Updating to deal with unrecognized last docs and moving event wrapper…

12c3010

… contents to writing analysis.

Merge pull request #44 from ArgLab/lastdocfix

afac01c

Lastdocfix

Updating running for observer server.

2a9e860

Fixing minor error in writing analysis.

1aa9b77

Debug code for looking at Google APIs, and in particular, Google Docs…

d5065b3

… APIs

Moved args into function Google API prototype, so we can use it as a …

088376c

…library

Starting to create a comprehensive Google API subsystem

b013e0a

Moving to data structure for Google routes

130fa06

Added initialization alert

0ce104b

Fixed sorting

6ceab44

Merge branch 'master' of https://github.com/ETS-Next-Gen/writing_obse…

ab02941

…rver

Google APIs now handled through a registration mechanism, and we're s…

da2c18c

…tarting to move rosters to use new API

Adding filesystem-based KVS, prototype zero

531f601

We can save AJAX requests. We might be able to replay them (not tested)

bef4d7b

Updates

7330ead

- Cleaned up lots of duplicated code - Added child support to possible selectors - Removed text from settings option, focusing on highlighting, we can revisit text as a component later - modularized pieces of dashboard

DrLynch and others added 20 commits May 23, 2023 01:10

Fixing load error on google.py startup to ablate unneeded code.

70fe55f

Fixing basic error in aggregator.

389eb34

Fixing new dashboard structure.

e9df676

Fix content script unload/reload

bf77d59

Minor tweak to service startup.

9a8fa8e

Making minor fix in module loading for new libraries.

cf5b844

Adding explicit cuda call.

99b2491

Updating Add script to deal with 12x bug on spacy install.

19e3724

Fixing the manifest issue for permissions. (#81)

66dc5ee

* Fixing the manifest issue for permissions. * Fixing manifest to reflect new version and to remove permissions that we do not make use of.

Rebasing from master and adding module updates.

bd94d74

Merge branch 'master' of https://github.com/ArgLab/ArgLab_writing_obs…

2020d09

…erver Dealing with existing conflicts dfuring merge.

Renamed functions & variables, added parameter definitions

b69f011

Cleaning up minor differences after rebase.

0dc12ef

Reduce gap between cache read and write

cda01af

Remove blank lines

d1c7d77

Merge branch 'master' into caching

d699db9

Merge branch 'master' into caching

08f7503

Merge Master into Caching

Add functionality to cache spacy docs

dad7ff3

Remove debug statements.

f2c90d1

sarthakbabbar3 requested review from DrLynch, bradley-erickson and pmitros June 16, 2023 15:08

sarthakbabbar3 added 2 commits June 20, 2023 23:54

Change to multi-line code for better readability and add comments.

13cab2e

Fix Linting.

8899e53

bradley-erickson reviewed Jun 23, 2023

View reviewed changes

Update spacy docs after indicator function calls

25b987d

DrLynch changed the title ~~Caching spacy doc~~ WIP: Caching spacy doc Jun 30, 2023

DrLynch force-pushed the caching_spacy_doc branch from 95980a2 to 25b987d Compare July 26, 2023 19:43

DrLynch pushed a commit that referenced this pull request Oct 20, 2023

Askgpt (#93)

7416793

* initial new layout for askgpt dashbaord * Documented code

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

WIP: Caching spacy doc #93

WIP: Caching spacy doc #93

Uh oh!

sarthakbabbar3 commented Jun 16, 2023

Uh oh!

bradley-erickson Jun 23, 2023

Uh oh!

bradley-erickson Jun 23, 2023

Uh oh!

bradley-erickson Jun 23, 2023

Uh oh!

bradley-erickson Jun 23, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants



		async def process_texts_serial(texts, options=None):
		async def process_texts_serial(texts, doc=None, options=None):



		async def process_texts_parallel(texts, options=None):
		async def process_texts_parallel(texts, doc=None, options=None):

WIP: Caching spacy doc #93

Are you sure you want to change the base?

WIP: Caching spacy doc #93

Uh oh!

Conversation

sarthakbabbar3 commented Jun 16, 2023

Uh oh!

bradley-erickson Jun 23, 2023

Choose a reason for hiding this comment

Uh oh!

bradley-erickson Jun 23, 2023

Choose a reason for hiding this comment

Uh oh!

bradley-erickson Jun 23, 2023

Choose a reason for hiding this comment

Uh oh!

bradley-erickson Jun 23, 2023

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants