Skip to content

Start shutdown with planner #25

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: lab
Choose a base branch
from
Open

Start shutdown with planner #25

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
5fc4b3a
add new high level functions and separate charging and idle
olaghattas Feb 8, 2025
6fa0d6b
add logic to read and write predicates to deal with rebooting
olaghattas Feb 8, 2025
28d630d
fixed reading and writing and updating problem file
olaghattas Feb 10, 2025
c5c0afb
fixed shutdown
olaghattas Feb 18, 2025
33da7dd
fixes
MarzanShuvo Feb 18, 2025
7de5a86
Update README.md
olaghattas Feb 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
256 changes: 45 additions & 211 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,33 +1,3 @@

## Simulator keyboard controls
`shift+O` : open door

`shift+M` : move to door

`shift+B` : move to bedroom

`shift+C` : move to couch

`shift+K` : toggle move to kitchen position 1/2

`shift+V` : take medicine

`shift+R` : go outside

`shift+S` : move to dining

`shift+J` : eat

'arrows' : nagivate camera

'shift+arrows' : move person around

`right-click and drag` : pan camera

`mouse wheel` : zoom in



## Managing source package dependencies
A source package must be build in order to use it. It is recommended to manage these packages using a .repos file. In this file, git repositories are listed in a .yaml file format and can be downloaded using vcs. To install vcs you can run:
```
Expand Down Expand Up @@ -58,7 +28,6 @@ Note, the above command must be run at the root of the ROS workspace.
Use Blender to view and edit mesh files. When exporting .obj files, make sure that Z is "up" and Y is "forward".
The values can be chosen from the export options.


# Smart Home Robot
The project is about i) designing a smart home equipped with a socially assistive robot (SAR) and serval
internet of things (IoT) devices and ii) evaluating the feasibility of using such a smart home to provide care-giving service for
Expand All @@ -75,207 +44,72 @@ elderly occupant and the safety of the home.

[[pdf]](http://cs.unh.edu/~tg1034/publication/shr_sajay.pdf)

# Install:
**Speech module**
Better TTS voices:
Download the new voice from [here](https://universitysystemnh-my.sharepoint.com/:u:/g/personal/pac48_usnh_edu/ERrsvRkJHx1Fve_Uv4RBRQ0BOGGKMvEZCGmGE4-R7GwuyQ?e=9730gE) then extract and copy it into `/usr/share/festival/voices/english`

Additional voices can be found here: http://www.festvox.org/packed/festival/2.5/voices/

**ffmpeg**
ffmpeg

**face module package**
`sudo apt-get install ros-kinetic-people-msgs`
`sudo apt-get install ros-kinetic-jsk-rviz-plugin`
`sudo pip2 install face_recognition`
`sudo pip2 install opencv-python`

**Speech module**
`sudo apt-get install ros-kinetic-sound-play`
Better TTS voice: https://ubuntuforums.org/archive/index.php/t-751169.html
Download the new voice from [here](https://universitysystemnh-my.sharepoint.com/:u:/g/personal/pac48_usnh_edu/ERrsvRkJHx1Fve_Uv4RBRQ0BOGGKMvEZCGmGE4-R7GwuyQ?e=9730gE) then extract and copy it into `/usr/share/festival/voices/english`

**primesense camera drive**
`sudo apt install libopenni2-dev`
`sudo apt install ros-kinetic-openni2-launch`
`sudo apt install ros-humble-depth-image-proc`
The primesense camera has to be connect to usb2.0 port

**Aria package(for rosaria)**
`sudo apt install libaria-dev`

**ROSPlan**
`sudo apt install ros-kinetic-mongodb-store`
ROSPlan: https://github.com/KCL-Planning/ROSPlan

**Pull and build SHR**
```bash
mkdir -p ~/catkin_ws/src
cd ~/catkin_ws/src
mkdir -p ~/smart-home/src
cd ~/smart-home/src
git clone [email protected]:AssistiveRoboticsUNH/smart-home.git
cd ~/catkin_ws
catkin build
# if you are using youcompleteme so need a compile database:
# catkin build --cmake-args -DCMAKE_EXPORT_COMPILE_COMMANDS=1
```

# Other pre-configuration:
**Grant usb port read permission**
`sudo usermod -a -G dialout $USER`
Have to reboot after.

**Laser Scan Ethernet Config**
On onboard labtop, set ethernet ip as 192.168.0.10

The default set for lms500 is 192.168.0.1, and we don't have to change this. The labtop ethernet have to be in the same ip domain, so anything similar to 192.168.0.x will work. Here we use 192.168.0.10.
This is a [reference artical in chinese](https://blog.csdn.net/zhuoyueljl/article/details/75244563) about the LMS500 laser.

**Set up for remote control**

Add this to .bashr or .zshrc:
* on board labtop
```bash
# need to add this to onboard labtop that runs roscore
export ROS_IP=0.0.0.0

# set to localhost for onboard labtop that runs roscore
export ROS_MASTER_URI=http://localhost:11311
cd ~/smart-home
colcon build --symlink-install

# IP of onboard labtop that runs roscore
export ROS_HOSTNAME=10.21.152.74
```

* remote control pc or labtop
```bash
# IP of onboard labtop that runs roscore
export ROS_MASTER_URI=http://10.21.152.74:11311

# IP of remote pc
export ROS_HOSTNAME=10.21.98,194
# if you are using youcompleteme so need a compile database:
colcon build --symlink-install --cmake-args -DCMAKE_EXPORT_COMPILE_COMMANDS=1
```
**Set up for audio and video resource**
`export ROS_WORKSPACE=/path/to/your/catkin_ws`

# Usage:

## Simulator:
**Bring up simulator:**
`roscore && roslaunch pioneer_shr pioneer_gazebo.launch`

**Mapping in gazebo:**
`roslaunch pioneer_shr pioneer_gazebo_mapping.launch`

**Autonomous navigation in gazebo:**
`roslaunch pioneer_shr auto_navigation_gazebo.launch`

**Visualize trajectory**
`roslaunch pioneer_shr trajectory_vis.launch`
(you may want to visualize the trajectory after mannually localize the robot)

## Real Robot mapping:
**ssh into onboard labtop**

**Bring up pioneer from onboard labtop:**
`roscore && roslaunch pioneer_shr real_mapping.launch`

**Start teleop keyboard from onboard labtop:**
`roslaunch pioneer_shr keyboard_ctrl.launch`

**Visualize from remote PC:**
`roslaunch pioneer_shr remote_vis.launch`

## Real Robot auto-navigation:
**ssh into onboard labtop**

**Bring up pioneer from onboard labtop via ssh:**
`roscore && roslaunch pioneer_shr auto_navigation_real_world.launch`

**Start teleop keyboard from onboard labtop via ssh:**
`roslaunch pioneer_shr keyboard_ctrl.launch`
# Setup planner to start and shutdown

**Visualize from remote PC:**
`roslaunch pioneer_shr remote_vis.launch`
Step 1:
add a cron job to clear out the intersection.txt file that include the predicates that need to be set true in the knowledge

## Auto-navigation trigger by sensor (obselet):
**Do all steps in auto-navigation**
run ''' crontab -e '''
and add
1 0 * * * > /path_to_shr_plan/include/shr_plan/intersection.txt

**Bring up simple_navigation_goal ros node**
`roslaunch pioneer_shr sensor_trigger_move2goal_real.launch` (real world)
`roslaunch pioneer_shr sensor_trigger_move2goal_gazebo.launch` (gazebo)
Step2:
in The keyword.txt add all the predicates that indicate that a low level protocol is successful
for example Medicine protolc is sucessful if either predicates (already_took_medicine) (already_reminded_medicine) are true so they should be added to the keyword.txt

## Face detection in gazebo:
**Do Autonomous navigation in gazebo**
Step3:

**start person sim**
`roslaunch person_sim init_standing_person.launch`
you need to set the two dictionaries protocol_type_ and keyword_protocol_ in action.hpp in the high_level_domain_Shutdown function.

**keyboard control for the person**
`roslaunch person_sim move_person_standing.launch`
this one should include all the protocols in :objects in the problem_high_level.pddl

**run face detection**
`roslaunch pioneer_shr face_detection_gazebo.launch`
const std::unordered_map<std::string, std::string> protocol_type_ = {
{"protocol name ", "Type"}
};
for example

## Face detection in real world:
**Do Autonomous navigation in real world**

**run face detection**
`roslaunch pioneer_shr face_detection_real.launch`

## Face recognition in real world:
**Do Autonomous navigation in real world**

**run camera on robot labtop**
`roslaunch pioneer_shr camera_real.launch`
(not needed if have face_detection running first)

**start face recognition on remote labtop**
`roslaunch pioneer_shr face_recognition_real.launch`

## Medcial Protocal in simulation:
**Do face detection and face recogniton in gazebo**

**run approach person service**
`roslaunch pioneer_shr action_service_gazebo.launch`

**run executive**
`rosrun pioneer_shr executive`

## Medcial Protocal in real world:
**Do face detection and face recogniton in real world**

**run approach person service**
`roslaunch pioneer_shr action_service_real.launch`

**run executive**
`rosrun pioneer_shr executive`

## Mid Night Protocal in real world:

**launch robot and run all service**
`roslaunch pioneer_shr shr_real.launch`

**launch face recognition on robot labtop**
`roslaunch pioneer_shr face_recognition_real.launch`

**run executive**
`rosrun pioneer_shr executive p2`
problem_high_level.pddl if looks like this
(:objects
living_room kitchen home outside dining_room bedroom bathroom - Landmark
am_meds pm_meds - MedicineProtocol
breakfast - FoodProtocol
)

then protocol_type_ should look like
const std::unordered_map<std::string, std::string> protocol_type_ = {
{"am_meds", "MedicineProtocol"},
{"pm_meds", "MedicineProtocol"},
{"breakfast", "FoodProtocol"}
};

## rosplan simple demo:
As for the keyword_protocol_ it maps the success keyword to the its protocol.
for example:

**launch robot and run all service**
`roslaunch pioneer_shr shr_real.launch`
(already_took_medicine) (already_reminded_medicine) are success keywords for am_meds and pm_meds
and (already_reminded_move) is a success for move_reminder

**launch planner**
`roslaunch rosplan_shr shr.launchp`
the keyword_protocol will look like this:

**run executive**
`rosrun pioneer_shr executive pddl`
const std::unordered_map<std::string, std::vector<std::string>> keyword_protocol_ = {
{"already_took_medicine", {"am_meds", "pm_meds"}},
{"already_reminded_medicine", {"am_meds", "pm_meds"}},
{"already_reminded_move",{"move_reminder"}},
};

## rosplan dry run on contigent-FF:

cd to catkin_ws/src/rosplan_shr
`rosrun rosplan_planning_system Contingent-FF -o ./common/domain_shr_conditional.pddl -f ./common/problem_shr_conditional.pddl`



Expand Down
2 changes: 0 additions & 2 deletions shr_parameters/params/shr_parameters.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,7 @@ shr_parameters:
}
eat_times: {
type: string_array,

default_value: [ "08h15m0s/08h45m0s" ],

description: "time that each protocol is triggered",
}
MedicineProtocols:
Expand Down
36 changes: 27 additions & 9 deletions shr_parameters_py/shr_parameters_py/shr_parameters.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,23 +28,17 @@ class __Instances:
LandmarksRobot = ["home", "designated_space"]
Persons = ["nathan"]
instances = __Instances()

class __Foodprotocols:
instances = ["breakfast", "lunch", "dinner"]
eat_times = ["06h45m0s/08h00m0s", "00h00m00s/00h00m00s", "00h00m00s/00h00m00s"]
eat_locations = ["kitchen", "kitchen", "kitchen"]
check_guide_to_succeeded_times = ["0h1m0s", "0h1m0s", "0h1m0s"]
remind_automated_food_at_times = ["0h10m0s", "0h10m0s", "0h10m0s"]
remind_automated_food_at_2_times = ["0h10m0s", "0h1m0s", "0h10m0s"]
instances = ["breakfast"]
eat_times = ["08h15m0s/08h45m0s"]
FoodProtocols = __Foodprotocols()

class __Medicineprotocols:
instances = ["am_meds", "pm_meds"]
take_medication_times = ["09h00m0s/10h00m0s", "21h00m0s/22h00m0s"]
MedicineProtocols = __Medicineprotocols()
class __Internalcheckreminderprotocols:
instances = ["internal_check_reminder"]
internal_check_reminder_times = ["08h15m0s/08h45m0s"]
internal_check_reminder_times = ["22h00m0s/23h00m0s"]
InternalCheckReminderProtocols = __Internalcheckreminderprotocols()
class __Practicereminderprotocols:
instances = ["practice_reminder"]
Expand Down Expand Up @@ -114,6 +108,14 @@ def update(self, parameters):
updated_params.pddl.instances.Persons = param.value
self.logger_.debug(param.name + ": " + param.type_.name + " = " + str(param.value))

if param.name == self.prefix_ + "pddl.FoodProtocols.instances":
updated_params.pddl.FoodProtocols.instances = param.value
self.logger_.debug(param.name + ": " + param.type_.name + " = " + str(param.value))

if param.name == self.prefix_ + "pddl.FoodProtocols.eat_times":
updated_params.pddl.FoodProtocols.eat_times = param.value
self.logger_.debug(param.name + ": " + param.type_.name + " = " + str(param.value))

if param.name == self.prefix_ + "pddl.MedicineProtocols.instances":
updated_params.pddl.MedicineProtocols.instances = param.value
self.logger_.debug(param.name + ": " + param.type_.name + " = " + str(param.value))
Expand Down Expand Up @@ -205,6 +207,16 @@ def declare_params(self):
parameter = updated_params.pddl.instances.Persons
self.node_.declare_parameter(self.prefix_ + "pddl.instances.Persons", parameter, descriptor)

if not self.node_.has_parameter(self.prefix_ + "pddl.FoodProtocols.instances"):
descriptor = ParameterDescriptor(description="food protocols", read_only = False)
parameter = updated_params.pddl.FoodProtocols.instances
self.node_.declare_parameter(self.prefix_ + "pddl.FoodProtocols.instances", parameter, descriptor)

if not self.node_.has_parameter(self.prefix_ + "pddl.FoodProtocols.eat_times"):
descriptor = ParameterDescriptor(description="time that each protocol is triggered", read_only = False)
parameter = updated_params.pddl.FoodProtocols.eat_times
self.node_.declare_parameter(self.prefix_ + "pddl.FoodProtocols.eat_times", parameter, descriptor)

if not self.node_.has_parameter(self.prefix_ + "pddl.MedicineProtocols.instances"):
descriptor = ParameterDescriptor(description="medicine protocols", read_only = False)
parameter = updated_params.pddl.MedicineProtocols.instances
Expand Down Expand Up @@ -296,6 +308,12 @@ def declare_params(self):
param = self.node_.get_parameter(self.prefix_ + "pddl.instances.Persons")
self.logger_.debug(param.name + ": " + param.type_.name + " = " + str(param.value))
updated_params.pddl.instances.Persons = param.value
param = self.node_.get_parameter(self.prefix_ + "pddl.FoodProtocols.instances")
self.logger_.debug(param.name + ": " + param.type_.name + " = " + str(param.value))
updated_params.pddl.FoodProtocols.instances = param.value
param = self.node_.get_parameter(self.prefix_ + "pddl.FoodProtocols.eat_times")
self.logger_.debug(param.name + ": " + param.type_.name + " = " + str(param.value))
updated_params.pddl.FoodProtocols.eat_times = param.value
param = self.node_.get_parameter(self.prefix_ + "pddl.MedicineProtocols.instances")
self.logger_.debug(param.name + ": " + param.type_.name + " = " + str(param.value))
updated_params.pddl.MedicineProtocols.instances = param.value
Expand Down
2 changes: 1 addition & 1 deletion shr_plan/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ install(TARGETS
)

install(
DIRECTORY launch pddl
DIRECTORY launch pddl include
DESTINATION share/shr_plan
)

Expand Down
Loading